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CHAPTER  1 


INTRODUCTION 


Background 

The  key  occupational  group  in  an  industrial  society 
is  embodied  in  that  of  management  and  managers  (Campbell  et 
al,  1970).  Management  has  been  defined  as  the  creation  and 
maintenance  of  an  environment  in  which  people  can  accomplish 
goals  efficiently  and  effectively.  These  organizational 
environments  are  directed  towards  the  achievement  of  organ¬ 
ization  goals,  integrating  human,  financial,  and  natural 
resources.  The  person  responsible  for  the  efficient  and 
effective  use  of  a  given  set  of  resources  is  the  manager 
(Albanese,  1978). 

In  today's  environment  of  high  inflation  rates  and 
cutbacks  in  federal  spending  the  Department  of  Defense  (DOD) 
is  pressed  to  maintain  the  efficient,  effective  organization 
necessary  to  achieve  its  goal  of  national  defense  with 
limited  resources.  In  order  to  insure  the  proper  use  of 
these  resources,  good  managers  must  be  obtained,  trained, 
and  retained.  The  selection  of  good  managers  is  greatly 
enhanced  by  insuring  that  the  selection  process  for  these 
managers  is  also  efficient  and,  especially,  effective. 

There  are  numerous  methods  of  manager  selection  avail¬ 
able.  Included  are  personal  data  sheets,  tests,  interviews, 
supervisory  ratings,  assessment  centers,  and  variants  or  com¬ 
binations  of  these  methods.  One  method  of  particular  interest 
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is  the  assessment  center.  Since  its  first  practical  industrial 
use  in  1956  at  the  American  Telephone  and  Telegraph  company 
(AT&T)  (Bray  and  Grant,  1966),  the  assessment  center  has 
achieved  general  acceptance  and  success.  The  average  validity 
of  the  assessment  center  in  the  prediction  of  managerial  suc¬ 
cess  is  about  as  high  as  the  maximum  achieved  by  other  trad¬ 
itional  methods  (Norton  and  Edinger,  1978). 

As  with  any  selection  method,  it  must  be  insured 
that  the  assessment  center  accurately  reflects  the  target  job 
being  measured.  This  is  content  validity.  In  addition,  the 
assessment  center  must  be  consistent  or  reliable  in  the  rat¬ 
ings  which  are  given  to  assessees,  or  internal  validity. 
Finally,  it  must  be  able  to  show  a  relationship  between  the 
rating  and  the  success  criteria  used,  external  or  predictive 
validity. 

Statement  of  the  Froblem 

The  Aeronautical  Systems  Division  (ASD)  at  ’.Vright- 
Patterson  AF3,  Ohio  conducted  as  assessment  center  in  1974 
and  1975*  This  assessment  center  was  designed  to  augment  the 
existing  Merit  Promotion  Plan.  Specifically,  it  would  be 
used  to  aid  in  the  evaluation  of  employees,  in  the  grades  of 
GS-9  and  above,  for  selection  to  first  level  supervisory  posi¬ 
tions  ( ASDRP  30-1,  1974). 

A  predictive  validity  study  of  this  assessment  center 
was  accomplished  in  1980  in  a  master's  thesis  effort  (Komar 
and  '.Vise,  1980).  Using  a  sample  of  size  N=143  of  those 
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cersonnel  who  attended  the  assessment  center,  four  criteria 
variables  were  compared  against  the  assessment  center  overall 
rating,  the  predictor  variable.  Correlation  coefficients  re¬ 
sulting  from  this  study  were  (1)  r=,27  with  the  number  of 
promotions  received  since  the  assessment  center,  (2)  r=.15 
with  salary,  (3)  r=.24  with  grade  (rank),  and  (4)  r=.28  with 
the  most  recent  merit  appraisal  ratings  for  those  sampled 
personnel.  Although  these  values  were  statistically  signifi¬ 
cant  at  less  than  the  .05  level,  they  are  not  as  strong  as  the 
median  correlation  coefficient,  r=.3?>  found  in  other  studies 
(Cohen  et  al,  1974)  (Byham  and  7/e ttengel,  1974).  A  need  to 
investigate  an  alternative  method  of  confirming  the  predictive 
validity  of  the  1974-1975  ASD  assessment  center  is  indicated. 

Objectives  of  the  Research 

The  overall  objective  of  this  thesis  is  to  further 
investigate  the  predictive  validity  of  the  1974-1975  ASD 
assessment  center.  Specifically  the  objectives  are: 

1.  Develop  a  technique  for  testing  the 
predictive  validity  of  assessment 
centers. 

2.  Implement  the  technique  for  the  ASD 
assessment  center  subjects. 

3.  Ascertain  application  value  of  the 
technique  to  future  assessment  centers. 

Scooe  and  Limitations 

This  thesis  is  limited  to  an  investigation  and 
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development  of  a  validation  technique  for  the  1974-1975  ASD 
assessment  center.  Only  those  personnel  v/ho  attended  this 
assessment  center  and  subsequently  achieved  supervisory  status 
are  utilized  in  this  study. 

Approach  to  the  Problem 

The  generation  of  alternative  criteria  was  aided  by 
the  examination  of  the  history  of  the  assessment  center  in 
general,  the  history  of  the  1974-1975  ASD  assessment  center 
specifically,  assessment  center  predictive  validity,  and  a 
comparison  of  various  predictors  and  criteria. 

The  criteria  used  are  a  combination  of  interview 
techniques  and  questionnaire  techniques  to  ascertain  various 
ratings  of  the  assessment  center  subject's  current  managerial 
performance.  Information  on  the  subjects  was  obtained  from 
their  respective  supervisors. 

The  content  of  the  questionnaire  consists  of  a  rating 
of  the  subject  on  the  same  14  managerial  dimensions  measured 
in  the  assessment  center.  In  addition,  a  measure  of  impor¬ 
tance,  or  criticalness,  of  those  14  dimensions  to  the  effec¬ 
tive  completion  of  managerial  jobs  is  included. 

The  content  of  the  interview  was  twofold.  First,  the 
raters  were  questioned  to  obtain  general  reactions  to  the 
assessment  center  and  to  obtain  a  measure  of  the  subject's 
perceived  overall  performance  effectiveness  in  the  current 
job.  Second,  the  interview  was  used  as  a  vehicle  to  insure 
the  rater  that  this  study  is  to  be  used  only  as  research  and 
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•.vill  have  no  impact  on  the  career,  salary,  or  Personnel  records 
of  the  subject  assessee,  This  is  accomplished  because  it  is 
felt  that  the  supervisor  would  tend  to  be  more  unbiased  and 
honest  in  his  rating  than  if  it  were  to  be  used  operationally 
(e.g.,  as  a  decision  input  of  the  promotion  process)  (Norton, 
1930c) . 

Summary 

This  chapter  has  provided  an  introduction  to  the 
research  subject,  identified  the  problem,  and  outlined  the 
objectives  and  limitations  of  the  study.  The  remaining  chap¬ 
ters  v/ill  cover  a  more  complete  background  of  the  subject, 
the  methodology  for  achieving  the  stated  objectives,  the  results 
and  analysis  of  the  study,  and  conclusions  and  recommendations. 
The  next  chapter  ’.vill  include  a  background  study  of  the  assess¬ 
ment  center  concept,  a  discussion  of  the  measurement  of  man¬ 
agerial  effectiveness,  and  a  review  of  assessment  center 
validation. 


CHAPTER  2 


ASSESSMENT  CENTERS  AND  VALIDITY 

History  of  the  Assessment  Center 

The  assessment  center  concept  originated  in  Germany 
in  ’Vo rid  7/ar  One  as  an  officer  selection  procedure.  Conse¬ 
quently,  early  usage  of  assessment  centers  was  limited  to 
military  uses.  In  World  7/ar  Two,  the  British  used  it  for 
officer  selection  (Huck,  1977)  and  the  United  States  used  it 
for  the  selection  of  intelligence  agents  (MacKinnon,  1977). 

Application  in  the  American  business  environment 
commenced  in  1956  with  a  Management  Progress  Study  conducted 
at  AT&T  (Bray  and  Grant,  1966).  Results  of  this  study  were 
used,  not  for  management  application  (e.g.,  promotion  or 
salary  decisions),  but  for  research.  Continued  development 
and  application  of  the  assessment  center  spread  from  use  in 
AT&T  to  ether  business  firms.  A  summary  of  some  of  these 
studies  is  shown  in  Tables  2-1 . 

The  first  operational  use  of  assessment  centers  in 
government  since  the  World  War  Two  study  for  intelligence 
agent  selection  occurred  in  the  Internal  Revenue  Service  (IRS) 
in  1969  (Byham  and  Wettengel,  1974).  (There  were  some  research 
efforts  performed  in  the  government  in  the  interim,  re: 
MacKinnon  et  al,  1958.)  Since  the  IRS  usage,  the  assessment 
center  has  been  used  by  the  U.S.  Forest  Service,  the  Federal 
Aviation  Agency,  the  Tennessee  Valley  Authority,  and  the 
U.S.  Postal  Service  to  name  but  a  few  (Cohen  and  Jaffee,  1973). 
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A  SUMMARY  OF  RESEARCH  STUDIES 


Additionally,  assessment  centers  have  been  used  in  several 
other  countries  such  as  Canada,  Australia,  Japan,  Brasil,  and 
South  Africa  (Huck  and  Bray,  1976). 

Uses  of  assessment  centers  in  DCD  include  the  U.3. 

Army  and  the  U.S.  Air  Force.  The  U.S.  Army  used  the  center 
as  a  method  of  prediction  of  performance  of  junior  officers 
as  well  as  the  identification  of  personal  and  career  devel¬ 
opment  areas  which  are  in  need  of  further  training.  In  the 
U.S.  Air  Force,  uses  have  been  primarily  restricted  to  Y/right- 
Patterson  AFB  activities  although  it  is  used  at  Squadron  Of¬ 
ficer's  School  at  Maxwell  AFB,  AL  as  a  training  aid.  7/right- 
Patterson  assessment  centers  have  been  oriented  to  the  selec¬ 
tion  of  civilian  personnel  for  position  promotions  or  identify 
cation  of  training  needs.  The  1974-1975  assessment  center  was 
directed  towards  selection  of  civilian  scientific  and  engi¬ 
neering  personnel  to  first  level  supervisory  positions.  The 
assessment  center  concept  was  used  again  in  1979  in  the 
Upward  Mobility  Program  (Norton  et  al,  1980a).  Assessment 
centers  are  still  considered  as  possible  methods  for  the 
identification  of  training  needs  of  civilian  personnel 
(Wissman,  1981). 

Due  to  its  success  and  value  (as  determined  by  the 
using  organization  or  agency),  it  appears  to  be  certain  that 
the  assessment  center  will  continue  to  be  a  desirable  instru¬ 
ment  for  management  selection  and  development. 
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The  first  major  U.S.  Air  Force  application  of  the 
assessment  center  was  accomplished  in  early  1974.  The  Air 
Force  Systems  Command  (AFSC)  sponsored  a  test  center  in  an 
effort  to  determine  the  feasibility  of  using  the  assessment 
center  concept  for  the  selection  of  U3AF  civilian  personnel 
for  first  level  supervisory  positions  (ASD/DPCU,  1973).  Tv/o 
research  and  development  organizations  at  '.Vright-Patterson  AF3 
Chic  were  chosen  as  the  subject  organisations,  the  A3D  Engi¬ 
neering  Directorate  and  the  USAF  Avionics  Laboratory  (AFAL) 
(Bryant,  1975c).  These  organizations  are  representative  of 
the  Pesearch  and  Development  (P&D)  activities  conducted  in  the 
USAF  in  general  and  AFSC  in  particular. 

A  number  of  alternatives  were  examined  by  AFSC  in  the 
determination  of  the  design  and  content  of  the  USAF  assessment 
center.  Existing  assessment  centers,  such  as  the  American 
Management  Association  center  (Bryant,  1975c),  were  considered 
for  direct  application.  In  addition,  management  consultation 
firms  specializing  in  the  design  of  assessment  centers  were 
examined  to  determine  the  best  alternative  for  USAF  use.  Due 
to  the  "...  unique  and  specialized  work  force  involved" 
and  unique  systems  with  which  they  worked  (Bryant,  1?75c),  it 
was  decided  to  design  an  assessment  center  unique  to  the  US^.F 
environment. 

The  firm  chosen  for  this  task  was  Assessment  Associate 

Inc.  of  Emerson  II. J.,  headed  by  Dr,  J.L,  Koses,  Dr.  Moses  is 

a  noted  authority  on  assessment  centers  having  been  involved 
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in  the  design  and  implementation  of  the  1967-1963  AT&T  assess¬ 
ment  center  and  reviewing  abdications  and  validities  cf 
assessment  centers  (Moses  '.vith  Cohen  et  al,  1974)  (Moses  and 
By ham,  1977). 

The  objectives  of  the  Air  Force  assessment  center  were 
listed  as  follows  Moses,  T 973 _ 7 : 

1.  Evaluate  supervisory  skills  needed  to 
succeed  as  a  first  level  supervisor  in 
AST  or  AL  /“Avionics  Laboratory_7. 

2.  Provide  both  management  and  each 
participant  with  a  profile  of  ind¬ 
ividual  strengths  and  weaknesses. 

3.  Stimulate,  where  applicable,  appropriate 
self  developmental  activities  that  might 
be  warranted. 

These  objectives  were  used  as  guidelines  in  the  design  of  the 
assessment  center. 

Dr,  Moses  identified  14  dimensions  to  be  used  in  the 
assessment  center.  These  dimensions  are  measures  of  managerial 
success  and  were  determined  through  a  survey  of  supervisors  in 
those  organizations  to  be  involved  in  the  assessment  center. 

The  dimensions  identified  for  use  and  their  definitions  are 
listed  in  Figure  2-1, 

The  assessment  center  was  designed  to  observe,  measure, 
and  evaluate  the  14  dimensions  (Moses,  1973).  The  assess¬ 
ment  process  included  three  phases,  (1)  the  Assessment  Phase, 
assessees  participate  in  exercises  and  interviews  (see  Figure 
2-2)  (for  a  breakout  of  the  dimensions  observed  in  each  exercise, 
see  Table  2-2),  (2)  the  Evaluation  Phase,  assessors  prepare 

1 1 
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,  Oral  Communications  C kills  (GC) :  To  what  extent  can  this 
individual  effectively  express  his  ideas  orally? 

2.  Leadership  (L) :  To  what  extent  is  this  individual  able 
to  get  others  to  work  together  effectively  in  a  grou~? 

3.  Tnergy  ( LIT ) :  To  what  extent  does  this  individual  maintain 
a  high  level  of  activity? 

4.  Empathy  (EK):  To  what  extent  does  this  individual  exhibit 
a  concern  for  others? 

5.  Forcefuiness  (FO):  To  what  extent  is  this  individual  able 
to  command  attention  from  others? 

6.  Persuasiveness  (PE):  To  what  extent  is  this  individual 
able  to  sell  his  point  of  view  to  others? 

7.  Flexibility  (FL):  To  what  extent  can  this  individual  ad¬ 
just  to  new  situations  easily? 

3,  Stress  Tolerance  (3T):  To  what  extent  is  this  individual 
able  to  respond  appropriately  to  stressful  conditions? 

9.  Risk  Talcing  (RT):  To  what  extent  is  this  individual  wil¬ 
ling  to  take  either  risk  or  responsibility  to  achieve 
objectives? 

10.  Acquiring  Information  (AI):  To  what  extent  is  this  indi¬ 
vidual  able  to  obtain  information? 

11.  Organising  Skills  (OS):  To  what  extent  can  this  individual 
plan  and  organize  the  work  of  himself  and  others? 

12.  Problem  Solving  (PS):  To  what  extent  can  this  individual 
easily  find  the  best  solution  to  a  problem? 

13.  Decision  Making  (DM):  To  what  extent  can  this  individual 
make  decisions  of  high"quality? 

14.  V/ritten  Communications  Skills  (WCS):  To  what  extent  can 
this  individual  effectively  express  his  ideas  in  writing? 


Figure  2-1 


ASD  ASSESSMENT  CENTER  DIMENSIONS 


Source:  Komar  and  '.Vise,  I960 
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1.  Interview :  This  exercise  provides  an  opportunity  for  the 
assessor  to  gather  inputs  directly  from  the  candidate  in 
a  one-on-cne  situation.  It  addresses  such  areas  as  job 
likes  and  dislikes,  career  Planning,  interests,  and  any 
other  areas  the  candidate  wishes  to  discuss.  Prior  to 
the  interview,  each  candidate  is  asked  to  com  "lets  a 
questionnaire  which  asks  for  information  regarding  back¬ 
ground,  interests,  likes  and  dislikes. 

2.  City  Council:  This  simulation  "is  an  assigned  role, 
leaderless  group  discussion.  Six  participants  role  :1 ay 
City  Council  members,  who  have  been  called  together  to 
determine  the  best  use  to  be  made  of  a  Federal  financial 
grant  of  $1,000,000  to  their  city.  Facn  ’council  member' 
has  information  on  the  needs  of  one  City  Department.  Tech 
participant  is  to  try  to  get  the  largest  appropriation 
possible  for  his  department  while  still  helping  the  group 
swiftly  and  fairly  accomplish  its  task"  (Development 
Dimensions,  1973). 

3.  Management  Problems:  This  simulation  is  a  leaderless 
group  discussion  with  no  assigned  roles.  The  six  candi¬ 
dates  are  to  act  as  a  group  of  consultants  asked  to  give 
recommendations  concerning  an  organization's  problems. 

The  group  must  come  up  with  a  written  recommendation  of 
a  suggested  course  of  action  for  each  of  four  problems. 

All  group  members  must  agree  on  and  initial  the  recommen¬ 
dation. 

4.  Supervisory  Task  Force:  This  simulation  is  a  leaderless 
grouo  discussion  with  no  assigned  roles.  The  six  candi¬ 
dates  are  to  act  as  a  special  task  force  assigned  to  de¬ 
cide  upon  the  attributes  needed  to  successfully  Perform 
a  first  level  supervisory  job,  Bach  candidate  is  given 
a  period  to  study  the  list  and  rank  the  attributes  (rlus 
.any  they  want  to  add)  in  their  order  of  importance  to  a 
first  level  supervisor.  The  candidates  must  then  meet 
together  and  prepare  a  written  report  listing  the  agreed- 
upon  attributes. 

5.  Research  Budget:  This  is  an  individual  exercise  in  which 
the  candidate  is  given  a  short  description  of  a  situatim 
demanding  an  immediate  decision.  The  candidate  is  asked 
to  play  the  role  of  a  newly  appointed  personnel  director 
v;hc  must  make  an  immediate  decision  and  Present  the 
reasoning  behind  it.  A  staff  member  is  assigned  to  :lay 
the  role  of  a  resource  person  who  will  answer  any  cuec- 
tions  the  candidate  wishes  to  ask. 


Figure  2-2 

Moure e:  Komar  and  Vise,  19;B 
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Crganisational  Problem:  This  is  an  individual  enercise 
in  which  the  candidate  is  asked  to  play  the  role  of  a 
special  task  force  member.  The  task  force  is  concerned 
with  improving  organisational  effectiveness.  The  candi¬ 
date  is  told  of  a  number  of  concerns  being  raised  which 
suggest  that  the  present  form  of  the  organisation  is 


The  candidates 
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reports  and  reach  a  group  decision  on  the  evaluation  of  each 
assessee,  and  (3)  the  Feedback  Phase,  the  assessee  receives  a 
personal  interview  on  center  performance  and  both  the  assessee 
and  management  receive  a  profile  of  the  assesses' s  strengths 
and  weaknesses. 

A  trial  run  of  the  ASD  assessment  center  was  conducted 
from  January  to  June  of  1974  involving  82  candidates  and  12 
assessors  (second  level  supervisors  trained  for  the  process). 
Most  of  those  involved  in  this  test  run  felt  that  the  assess¬ 
ment  center  should  be  continued.  A  study  accomplished  later 
confirmed  that  94*;o  of  the  managers  felt  that  information  re¬ 
ceived  from  the  assessment  center  was  at  least  marginally 
useful  (Friedman,  1975). 

Based  on  the  information  from  the  test  run  it  was 
decided  to  implement  the  assessment  center  on  an  operational 
basis.  Formal  establishment  was  announced  in  July  1974  by 
Lt  Gen  James  Stewart,  then  Commander  of  A3D.  In  addition, 

",  .  .it  was  recommended  that  the  Assessment  Centers  be  ex¬ 
panded  to  other  AFSC  organizations"  and  to  career  fields  other 
than  scientific  and  engineering  (ASDPP  50-1,  1974).  The 
information  received  from  the  assessment  was  formally  incorpo¬ 
rated  into  the  Merit  Promotion  Plan,  ASD  R4C-4,  on  13  January 
1975  (Bryant,  1975a). 

The  assessment  center  was  discontinued  in  December 
1975  (Clayton,  1975).  This  was  due  to  a  Judge  Advocate  de¬ 
cision  that  there  was  ".  .  .no  authority  (within  the  meaning 


of  the  Frivacy  Act)  for  collecting,  maintaining  or  using  such 
records  as  are  kept  by  the  Assessment  Center  ^“DiPasqualucci, 

197 5_7." 

In  all,  284  civilians  v;ere  assessed  in  the  assess¬ 
ment  center  which  included  nine  sessions  in  the  trial  run 
and  16  sessions  in  the  operational  run.  A  revision  of  the 
Merit  Fromotion  Plan  in  1977  deleted  the  assessment  center 
as  a  factor  in  the  determination  of  promotability  ('.Vissman, 
1931). 

The  content,  internal,  and  predictive  validities  of 
the  assessment  center  were  examined  in  a  recent  master’s 
thesis  (Komar  and  '.Vise,  1980).  Cn  content  validity  they 
found  that  "...  based  on  the  thorough  job  analysis,  assess¬ 
ment  center  design,  training,  trial  run,  and  evaluation,  the 
A3D  Management  Assessment  Center  appears  to  possess  content 
validity."  The  internal  validity  was  tested  through  correla¬ 
tion  of  the  dimensions,  inter-rater  reliability,  factor  anal¬ 
ysis,  and  regression  of  dimensions  with  the  overall  Assess¬ 
ment  center  rating.  Their  findings  indicated  that,  in  general, 
the  assessment  center  information  was  reasonable  and  con¬ 
sistent.  Negative  findings  were  identified  by  Komar  and  '.Vise 
£  1 980_7  as  follows: 

1 .  Acquiring  information  should  be  excluded 
from  the  interview.  The  correlation  of  this 
dimension  with  the  final  dimension  rating 
indicates  that  very  little  weight  was  given 
to  this  dimension  by  the  assessors  in  that 
exercise. 

2,  The  organizational  problem  exercise  needs 


to  be  examined  for  possible  problems  in 
the  measurement  of  dimensions  within  that 
exercise.  Inter-rater  reliabilities  for 
all  four  dimensions  within  that  exercise 
were  considerably  lower  than  for  any 
other  exercise. 

3.  Definition  problems  exist  for  the  stress 
tolerance  dimension.  This  was  indicated 
first  by  the  low  inter-rater  reliability 
and  second  by  reviewing  the  assessor 
'worksheets.  In  some  cases,  the  same 
assessee  was  given  both  a  zero  and  a  five 
rating  by  different  assessors  because 
no  stress  was  observed. 

The  predictive  validity  was  ascertained  through  cor¬ 
relation  analysis  of  the  14  dimensions  and  overall  assessment 
center  rating  with  four  indicators  of  managerial  success. 

The  criteria  chosen  as  indicators  were  the  assessee' s  present 
grade,  present  salary,  most  recent  merit  appraisal  rating, 
and  the  number  of  promotions  received  since  the  assessment 
center.  Resultant  correlation  coefficients  are  shown  in 
Table  2-3.  Based  on  these  results,  Komar  and  '.Vise  determined 
that  as  the  coefficients  were  statistically  significant,  the 
assessment  center  was  valid  as  a  predictive  instrument. 

(One  minor  problem  found  in  predictive  validity  was  the 
insignificance  of  the  Empathy  dimension.  Komar  and  '.Vise 
recommended  deletion  of  this  dimension.) 


Predictors  and  Criteria  of  Managerial  Success 

Overview.  "Managers  are  accountable  for  performance 
.  .  ,  .  the  central  idea  for  managers  to  keep  in  mind  about 
performance  is  that  it  can  always  be  more  efficient  and  more 
effective  ^“Albanese,  1973— 7."  Since  the  manager  is  responsible 
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TA3LS  2-3 


CORRELATIONS  OF  FINAL  DIMENSION  AND  OVERALL  ASSESSMENT  RATINGS 


'.71 TI!  INDICATORS 

OF  MAN AG  1 

SRIAL  SUCCESS 

(N=143 

except  as 

noted) 

Dimensions  Promotions 

Salary 

.\pD*** 

Grade 

Mean 

CC 

.18* 

.26 

.51 

.52 

.27 

LE 

.25 

.18* 

.22 

.25 

.23 

EN 

.23 

.09** 

.19* 

.13* 

.17 

EM 

.00** 

-.10** 

.07** 

-.03** 

-.02 

FG 

.27 

.13** 

.20* 

.22 

.21 

PE 

.23 

.09** 

.20* 

.22 

.19 

FL 

.19* 

-.01 ** 

.11** 

.12** 

.10 

ST 

.20 

.15* 

.27 

.24 

.22 

RT 

.20 

.02** 

.28 

.14* 

.16 

A I 

.19* 

.07** 

.11** 

.16* 

.13 

C3 

.17* 

.15* 

.22 

.22 

.19 

FS 

.25 

.11** 

.24 

.23 

.21 

DM 

.29 

.09** 

.24 

.21 

.21 

v;c 

.19* 

.13* 

.29 

.25 

.23 

Mean 

.20 

.10** 

.21 

.20 

Overall 

Assessment 

.27 

.15* 

.28 

.24 

***  :r  =  1 10 

Significance:  ?  <  .01  except  as  noted 

*  ,01<  p  <  .05 
**  P  >-.05 

3ee  Figure  2-1  for  explanation  of  dimensions 

Source:  Komar  and  V/ise,  1930 
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for  performance,  then  one  measure  of  managerial  success  is 
the  attainment  of  an  acceptable  level  of  performance.  An 
acceptable  level,  or  levels,  is  in  a  large  cart  det^i  ined  by 
the  organisational  policy  and  goals.  One  author  defined 
effective  managerial  job  behavior  as  follows: 

Any  set  of  managerial  actions  believed  to  be  optimal 
for  identifying,  assimilating,  and  utilising  both  internal 
and  external  resources  toward  sustaining,  over  the  long 
term,  the  functioning  of  the  organisational  unit  for  which 
a  manager  has  some  degree  of  responsibility  ^“Campbell 
et  al,  1970__7. 

Effective  behavior  and  acceptable  performance  are  ascertained 
by  conducting  an  analysis  of  the  job  in  question.  This  in¬ 
volves  definition  of  a  job  description  (the  responsibility 
and  job  structure  relationships)  and  job  specifications  (job 
characteristics  and  qualifications).  These  are  formulated  to 
serve  as  a  basis  for  choosing  performance  criteria.  "A  meas¬ 
ure  of  any  given  manager's  effectiveness  should  be  based  on  a 
careful  definition  of  the  total  domain  of  his  job  responsi¬ 
bilities  ...  (/“Campbell  et  al,  1970 _7."  Those  measures 
which'  are  used  to  forecast  how  a  person  will  do  in  a  given 
job  environment  are  known  as  predictor  variables.  Those 
measures  which  are  used  to  reveal  hew  a  person  is  currently 
doing  in  a  given  job  environment  are  known  as  criteria  varia¬ 
bles.  In  determining  what  measures  to  use  as  predictors  or 
criteria,  an  element  of  objectivity  and  job  specificity  must 
be  maintained.  As  the  measures  become  more  vague  with  respect 
to  the  job  in  question  end  the  organization,  the  accuracy  of 
these  measurements  becomes  questionable. 


Predictors 


Predictors  are  used  to  predict  how  a 


person  will  do  or  his  potential  to  do  something  in  the  future. 
The  methods  available  for  the  prediction  of  managerial  success 
are  varied  in  number,  style,  and  validity. 

The  most  concrete  method  of  measurement  is  the 
paper  and  pencil  test.  Tests  come  in  a  variety  of  formats 
and  contents  to  measure  different  attributes  of  a  person  or 
suitability  for  a  specific  job.  Paper  and  pencil  testing  has 
come  under  legal  observation  because  of  the  suspicion  that 
such  tests  may  be  unfair  to  minority  groups  and  women  (Huck 
and  Bray,  1976). 

Mental  ability  tests  comprise  a  major  component  of 
testing  in  general.  These  are  usually  in  the  form  of  achieve¬ 
ment  tests,  measuring  past  accomplishments,  aptitude  tests, 
designed  to  predict  future  potentialities  of  a  person,  and 
proficiency  tests,  measuring  the  attained  skill  in  a  task. 

A  literature  review  by  Norton  and  Edinger  (1978)  found  va¬ 
lidity  coefficients  of  "moderate"  value  (r=.20s)  for  mental 
ability  tests.  They  later  conclude  that  these  tests  are  the 
only  traditional  predictive  method  (as  opposed  to  'new' 
methods,  such  as  the  assessment  center)  which  was  consist¬ 
ently  valid.  However,  a  number  of  studies  have  shown  that 
assessment  center  ratings  explain  10%  to  2.0%  greater  criterion 
variance  than  tests  alone  (Bray  and  Grant,  1966)  (Bray  and 
Campbell,  1963)  (Carleton,  1970)  (Pinley,  1970)  (Moses,  1971) 
(7/ollowick  and  McNamara,  1 969 )  • 
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Another  type  of  test  is  the  perscnality/interest  test. 
Personality  tests  reveal  characteristics  of  a  personality  that 
nay  affect  a  person's  success  in  adjusting  to  the  work.  In¬ 
terest  tests  help  a  person  identify  just  what  type  of  work 
that  person  likes  (Hassie  and  Douglas,  1977).  A  literature 
review  by  Horton  and  Edinger  (1978)  concluded  that  these 
tests  are  generally  less  consistent  in  results  achieved 
than  mental  ability  tests;  validation  results  were  nixed. 

Norton  and  Edinger  also  found  nixed  results  in  vali¬ 
dation  of  projective  personality  tests.  These  tests  attempt 
to  assess  a  person  through  his  or  her  fantasies  and  inter¬ 
pretations  (Szilagyi  and  Wallace,  1980).  Disadvantages 
noted  with  these  tests  are  the  high  expense  involved  and 
legal  proglens  due  to  low  face  validity. 

Besides  testing,  another  predictive  technique  is  the 
interview.  The  interview  process  involves  questioning  the 
individual  to  be  rated  on  various  aspects  of  education,  ex¬ 
perience  and  general  beliefs  in  order  to  ascertain  some 
measure  of  how  that  person  will  operate  in  the  proposed  job 
position.  The  complexity  of  the  interview  can  range  from 
simple,  unstructured  (no  set  agenda  or  questions)  one  to  one 
conversations  to  organized,  highly  structured  panel  interviews. 
A  Norton  and  Edinger  literature  review  led  to  the  conclusion 
that,  standing  alone,  the  interview  is  not  a  valid  oredictor. 
However,  the  incorporation  of  the  interview  into  the  ATA-T 
Assessment  Center  led  to  increased  validity  of  that  model 
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(3ray  and  Grant,  1966). 

The  use  of  personal  data  sheets,  or  biographical  in¬ 
formation,  in  essence  an  interview  on  caper,  has  proven  to 
be  a  good  predictor  of  managerial  success  (Campbell  et  al, 
1970).  Its  use,  however,  is  limited  due  to  the  confidenti¬ 
ality  of  the  information  (Norton  and  Fdinger,  1973). 

Supervisory  ratings,  those  ratings  derived  from  the 
person's  supervisor,  have  been  a  common  method  of  prediction 
in  both  civilian  and  military  organisations.  A  study  by 
Norton,  3alloun,  and  Konstantinovich  (1930b)  concluded  that 
supervisory  ratings,  by  themselves,  are  not  a  good  credictor 
Ratings  so  used  are  subject  to  uncertainties  and  distortions 
sc  as  to  make  such  ratings  suspect  (Rowland,  1970).  In  addi 
tion,  there  are  many  common  supervisory  rating  errors  which 
include  central  tendencies,  leniency,  strictness,  and  halo 
errors  (Szilagyi  and  7/allace,  1980). 

Finally,  a  variant  to  supervisory  ratings  are  peer 
ratings.  Peer  ratings  are  overall  ratings  of  managerial  sue 
cess  obtained  from  a  person's  cowcrkers.  Although  generally 
subject  to  the  same  deficiencies  (as  an  accurate  predictor) 
as  supervisory  ratings,  oeer  ratings  have  been  found  to  be 
better  as  predictors  (Campbell  et  al,  1970).  This  has  been 
attributed  to  the  fact  that  peers  have  a  better  chance  of 
observing  the  rated  individual  both  in  different  situations 
and  the  frequency  of  observation. 

Criteria.  Criteria  are  measures  of  managerial  effec 
iveness,  or  how  a  person  is  doing  in  the  current  job.  As 
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v;ith  predictors,  criteria  should  encompass  all  important 
aspects  of  the  job  in  question.  Again,  as  with  predictors, 
there  are  various  types  of  criteria. 

Cne  type  of  criteria  is  known  as  behavior  observation. 
This  is  the  defining  of  those  managerial  behaviors  necessary 
to  effective  job  fulfillment  (usually  accomplished  in  a 
thorough  job  analysis)  and  the  subsequent  rating  of  those 
behaviors.  Problems  v/ith  the  use  of  this  criteria  are  (1) 
the  possibility  of  using  a  deficient  job  analysis  (necessary 
behaviors  either  poorly  defined  or  omitted  altogether),  (2) 
a  difficulty  in  assuring  objectivity  among  raters  (deviations 
include  central  tendencies,  leniency,  strictness,  and  halo 
rating),  and  (3)  the  lack  of  consistency  among  raters  (dif¬ 
ferent  raters  reporting  a  single  occurence  differently) 
(Campbell  et  al,  1970). 

A  second  type  of  criteria  are  objective  measures, 
measuring  the  effectiveness  of  the  work  environment  created 
by  the  manager.  Included  in  this  category  are  methods  such 
as  the  direct  measurement  of  physical  aspects  of  a  job,  for 
example,  quantity  output  from  the  organisation,  absence  rates, 
and  turnover  rates.  Another  objective  measure  is  a  job  atti¬ 
tude  survey,  given  to  subordinates,  which  measures  their  feel¬ 
ings  about  the  job.  These  feelings  are  then  translated  into 
an  effectiveness  measure  for  the  organization.  A  difficulty 
found  with  objective  measures  is  that  they  tend  to  focus  on 
only  a  few  of  the  necessary  managerial  behavior  aspects  and, 
■thus,  represent  a  limited  effectiveness  or  success  rating 
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(Campbell  et  al,  1970) 


The  remaining  criteria  fall  under  the  general  cate¬ 
gory  of  global  ratings.  Global  ratings  measure  a  bread  spec¬ 
trum  of  managerial  behavior  in  a  single  rating.  If  the  sample 
size  io  small  enough  and  the  personnel  to  be  rated  can  be 
evaluated  as  a  part  of  a  grour,  ranking  may  be  used.  This 
method  involves  arranging  the  personnel  in  a  hierarchy  of 
effectiveness,  the  most  effective  personnel  first.  As  the 
number  of  personnel  to  be  rated  increases,  however,  this 
method  becomes  less  feasible.  Other  global  ratings  are 
salary  and  level  attained  in  the  organization  (these  are 
usually  adjusted  for  time  in  the  organization).  These  methods 
cover  a  number  of  managerial  behavior  aspects  in  that  they 
incorporate  the  sum  of  all  past  supervisory  judgements  of 
the  individual  in  the  salary  and  promotion  recommendations 
made  by  those  supervisors.  However,  this  is  also  a  factor 
which  can  confound  results  using  salary  and  level  in  that 
luck  (in  having  a  good  or  poor  rater  throughout  the  career) 
plays  a  part.  A  problem  found  with  global  ratings  in  general 
is  that  it  may  be  difficult  to  ascertain  if  the  rating  is 
confounded  by  information  which  is  irrelevant  to  the  job. 
Irrelevant  factors  which  may  have  an  impact  are  the  physical 
bearing  and  friendliness  of  the  rated  individual  (e.g.,  some¬ 
one  possessing  large  amounts  of  these  factors  may  tend  to  be 
rated  higher  than  his  true  effectiveness) (Camobell  et  al,  1970). 

Summary.  Whereas  predictors,  e.g.,  assessment  cen¬ 
ters,  are  used  to  predict  how  a  person  will  do  or  his  potential 
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to  do  something  in  the  future,  criteria  are  measures  of  man¬ 
agerial  effectiveness  against  which  the  predictors  may  be 
compared.  Predictors  make  a  forecast  of  how  a  person  will 
do;  criteria,  taken  at  a  later  date  than  the  predictors, 
measures  how  a  person  is  doing.  A  comparison  of  the  two 
gives  a  measure  of  the  accuracy,  or  validity,  of  the  fore¬ 
cast  made  with  the  predictor. 


Predictive  Validity 

An  evaluation  technique  may  be  conceptually  sound, 
but  if  the  technique  cannot  accurately  predict  the  desired 
element  (e.g.,  managerial  success),  then  it  is  cf  little 
value, 

A  traditional  measure  of  the  validity  of  any  given 
selection  or  evaluation  method  is  the  correlation  coefficient 


(symbolized  ' r ' )  (Horton  and  Sdinger,  1973).  This  coefficien 
measures  the  strength  of  the  relationship  between  variables 
(Harnett,  1975)#  Higher  coefficient  values  indicate  a 
stronger,  more  predictive  relationship.  In  the  A Z1  assess¬ 
ment  center,  the  center  ratings  are  the  predictors,  and  the 
measures  of  managerial  success  are  the  criteria  (e.g.,  sal¬ 
ary  level) . 


.Studies  of  various  assessment  centers  have  resulted 
in  correlation  coefficients  ranging  from  r=.01  to  r=.65 
(Finley,  1970)  depending  on  the  criterion  used.  Reviews  of 
19  assessment  centers  by  Cohen,  Moses,  and  3yham  (1974)  and 
of  22  centers  by  Byham  and  '.Vettengel  (1974)  both  resulted  in 
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median  correlation  coefficients  of  r=.37.  Dr.  Moses  indicated 
that  the  ADD  assessment  center  should  prove  to  have  a  predic¬ 
tive  validity  v/ell  above  r=.50  '.vhile  the  ASD  Director  of  the 
assessment  center  expected  to  achieve  a  correlation  with  iob 
success  criteria  within  the  approximate  range  of  r=,40  to 
r=.60  (Bryant,  1975d). 

A  review  of  assessment  center  validity  studies  and 
the  relationship  between  the  correlation  received  and  the 
criterion  used  are  summarized  on  Table  2-4.  Those  studies 
by  Komar  and  '.Vise  were  on  the  1974-1975  ASD  assessment  center. 
The  statistical  significance  of  the  correlation  values  is  net 
considered  on  this  table.  This  is  dependent  on  the  sample 
size. 

Taking  a  simple  situation  in  which  a  person  is  truly 
capable,  or  truly  incapable,  of  performing  a  managerial  job, 
the  assessment  center  will  make  the  correct  recommendation 
for  that  person  (i.e.,  recommend  if  capable  and  not  recommend 
if  not  capable)  a  percentage  of  the  time  and  be  incorrect  the 
remainder  of  the  time.  A  formula  to  determine  the  relation¬ 
ship  between  the  correlation  coefficient  and  the  percentage 
of  correct  recommendations  in  a  dichotomous  (Correct/Incorrect) 
situation  was  derived  by  Tate  and  Clelland  (1957).  This  for¬ 
mula  is  graphically  illustrated  in  Figure  2-3. 

As  can  be  seen  on  the  graph,  a  correlation  coefficient 

of  r=0  is  equivalent  to  pure  chance,  e.g.,  a  fli  :  of  a  fair 

coin.  As  the  correlation  coefficient  increases  in  value,  the 

percentage  of  correct  decisions  increases  at  a  faster  rate. 
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iUMMARY  OF  ASSESSMENT  CENTER  CORRELATION.' 
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Therefore,  although  a  given  correlation  coefficient  may  be 
statistically  significant  given  the  sample  size,  it  is  al¬ 
ways  possible,  and  desirable,  to  decrease  the  chance  of  an 
incorrect  desision. 

Summary 

This  chapter  has  described  the  history  of  the  assess¬ 
ment  center  in  general  and,  specifically ,  the  A5D  assessment 
center,  the  subject  of  this  study.  In  addition,  it  reviewed 
the  concepts  of  predictors  and  criteria  of  managerial  effec¬ 
tiveness.  Finally,  the  validity  issue  and,  specifically, 
predictive  validation  of  assessment  centers  was  discussed. 

The  next  chapter  outlines  those  procedures  used  to  meet  the 
stated  objectives. 
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CHAPTER  3 


Overview 

The  methodology  used  in  this  thesis  effort  was 
structured  so  as  to  best  meet  the  stated  objectives.  The 
1974-1975  A3D  assessment  center  resulted  in  data  used  to 
predict  the  managerial  success  for  civilian  personnel  being 
considered  for  promotion  to  the  first  level  surervisory 
position.  Using  this  data  as  the  predictor  variables,  a  set 
of  questions  and  ratings  were  derived  to  act  as  criteria. 

The  criteria  data  were  statistically  compared  with  the  predic 
tor  data  to  measure  the  predictive  validity  of  the  A CD  assess 
ment  center. 

Statistical  analyses  were  accomplished  using  the 
Statistical  Package  for  the  Social  Sciences  (SPSS)  (Hie  et  al 
1975).  The  Aeronautical  Systems  Division  CDC  6600  computer 
was  used  to  run  all  required  programs. 

Previous  Studies 

There  has  been  only  one  comprehensive  research  study 
of  the  1974-1975  ASD  assessment  center  and  its  predictive 
validity.  As  stated  in  Chapters  1  and  2,  this  was  a  thesis 
completed  by  Komar  and  V/ise  (1980).  Criteria  selected  for 
use  in  that  study  were  (1)  salary,  (2)  number  of  promotions 
since  the  assessment  center,  (3)  grade  (GS  level),  and  (4) 
most  recent  appraisal  rating.  In  addition  to  a  study  of  the 
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predictive  validity  of  the  assessment  center,  Komar  and  '.Vise 
conducted  a  thorough  internal  and  content  validity  study  of 
the  assessment  center.  As  these  subject  areas  have  already 
received  sufficient  research  coverage,  they  were  not  addressed 
in  this  thesis  effort.  Results  of  the  Komar  and  '.Vise  study 
are  as  shown  in  Chapter  2. 

Nature  of  the  Sample 

A  total  of  239  civilian  personnel  participated  in  the 
1974-1975  AoD  assessment  center.  A  total  of  25  centers  were 
run,  with  each  center  split  into  two  groups  of  six  assessees 
each.  The  first  nine  centers  encompassed  the  test  program 
whereas  the  final  16  centers  were  operational.  All  assessees 
were  in  scientific  and  engineering  related  career  fields  and 
held  a  grade  of  GS-9  to  GS-14.  The  target  job  of  the  assess¬ 
ment  center  was  selection  to  the  first  level  of  supervision 
(Hoses,  1975)  (ASD/DPCTJ,  1975)  (Stewart,  1974).  Of  the  total 
of  289  civilian  personnel  who  participated  in  the  assessment 
center,  105,  or  56%,  have  achieved  supervisory  status.  A 
summary  of  the  supervisory  level  for  these  105  is  shown  in 
Table  5-1.  Definitions  of  the  supervisory  level  titles  are 
shown  on  Figure  5-1 • 

As  the  assessment  target  level,  first  level  super¬ 
visors,  represents  45%  of  all  supervisors  and  only  16%  of  the 
entire  assessee  population,  it  was  decided  to  include  all 
assessees  having  achieved  any  supervisory  status.  Although 
not  true  supervisors  (in  that,  for  the  most  wart,  they  have 
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Schedule  supervisors  be- 
3  subordinates  and  all 


.  First  level  of  Supervision:  Foremen  and  IF  supervisors 
at  lowest  level  of  continuing  supervision  over  $  cr  more 
full-time  subordinates. 

.  Program  Management :  Employees  covered  by  10  U.3.C.  15c1 
or  in  grades  G3-12  through  GO- 1 3  who  have  advisory  re¬ 
search,  scientific,  engineering  and/or  comolete  or  limited 
"rcgram  responsibilities,  i.e.,  usually  wi t h  none  or  not 
more  than  tv/o  non- supervisory  subordinates  and  normally 
no  subordinates. 

.  Executive  Management:  Employees  covered  by  10  U.3.C.  1  pc  1 
or  General  'Schedule  Employees  in  grades  GO-15  through  13 
v:ith  full  colonel  or  above  equivalent  AFOC.  Fas  complete 
program  responsibility  and  substantial  continuing  super¬ 
visory  responsibilities,  i.e.,  normally  supervises  at 
least  two  subordinate  supervisory  levels,  or  at  least 
three  subordinate  managers,  supervisors,  or  non-su  ervisory 
personnel . 

.  Gecond  level  of  supervision:  V/ age  Schedule  or  General 
Schedule  Supervisor 

a.  whose  -primary  role  is  assuring  accomplishment  of  the 
work  assigned  to  the  activity  through  at  least  one  sub¬ 
ordinate  level  of  full-time  supervision 

b.  but  who  does  not  meet  the  requirement  for  managerial 
positions  as  defined  by  F?H  Letter  412-2,  29  January  1°?4. 

.  First  Managerial  Level:  Employees  whose  positions  meet 
the  requirements  for  identification  as  managerial  as 
defined  by  PPM  Letter  412-2,  29  January  1974  but  are  be¬ 
low  the  range  for  Executive  Management  (Code  5). 


Figure  3-1 

EXPLANATION  C?  SUPERVISORY  LEVEL  ^PKG 


Ecurce :  AOE/wwq"' 

* 'I  AO- 50J, 
51  May  1070 


Part-time  Supervision:  General 
low  G.3-12  level  with  fewer  than 
York  Leaders. 
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no  subordinates),  program  managers  were  included  to  increase 
the  potential  data  base.  Thses  individuals  are  important 
resource  managers  and  usually  must  interact  with  other  people 
to  significant  degree. 


TABLE  3-1 


A  CC  T?C!  <7  T?7? 

tioO  .jO*j  uJj 


SUPERVISE BY  LEVEL 


IN  1981 


Level 

V 

x  • 

% 

Fart  Time  Supervision 

2 

2 

First  Level  Supervision 

46 

45 

Second  Level  Supervision 

6 

6 

Program  Manager 

35 

34 

First  Managerial  Level 

4 

4 

Executive  Management 

10 

Q 

Totals 

103 

100 

An  attempt  was  made  to  contact  all  103  supervisory 
assessees.  Those  contacted  were  informed  of  the  subject  and 
nature  of  the  research  effort.  It  was  then  reauested  that 
they  verify  that  they  were  vailing  to  be  included  as  a  research 
subject.  Finally,  if  the  assessees  proved  cooperative,  it  was 
requested  that  they  give  their  permission  for  an  interview 
and  questionnaire  on  their  current  managerial  effectiveness 
to  be  given  to  their  supervisors. 

After  elimination  from  consideration  those  assessees 
and  assessee  supervisors  who  were  on  extended  TOY,  on  extended 
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leave,  and  who  refused  to  participate  in  the  study,  there 
remained  67  assessees,  These  6?  assessees  represent  a  sampl 
of  the  Komar  and  ’.Vise  study  group,  as  well  as  additional  as¬ 
sessees  from  the  original  assessment  center  group.  These 
additional  personnel  (23,  or  34,3  of  this  sample)  Participate 
in  the  nine  test  program  assessment  centers  which  Komar  and 
'.Vise  did  not  study,  A  summary  of  the  supervisory  levels  of 
these  67  is  shown  in  Table  3-4,  The  major  organizations  for 
which  the  67  assessees  worked  are  shown  on  Table  3-3,  A  sum 
mary  of  the  grade  level  of  the  67  assessees  is  shown  in  Tabl 
3-4,  A  final  point  of  consideration  is  that  all  but  one  of 
the  67  assessees  are  male, 

TABLE  3-2 

SUPERVISORY  LEVEL  OF  ASSESSEES  CHOSEN  FOR  STUDY 


Level  N  3 


Fart  Time  Supervision  2  3 
First  Level  Supervision  32  US 
Second  Level  Supervision  4  6 
Program  Manager  22  33 
First  Managerial  Level  1  1 
Executive  Management  6  9 


Totals  67  TOO 


The  immediate  supervisors  of  the  assessees  were  con¬ 
tacted  and  interviewed  concerning  the  managerial  effectivene 


of  the  assesses  in  their  current  jobs.  .'/hen  the  immediate 
supervisor  could  not  be  contacted,  when  possible,  someone 
senior  to  the  assessee,  yet  still  familiar  with  that  assesses' 
"erfcrmance  (e.g.,  a  former  supervisor),  was  contacted  and 
interviewed. 


.10  0  i^OO  1j ±-j 

TABLE  3-3 

0 RGANI ZATICNAL  MAKEUP 

Organization 

N 

rf 

,0 

ASD 

37 

55 

AF7/AL 

25 

37 

FTD 

5 

3 

Totals 

67 

100 

TABLE  3-4 

DISTRIBUTION  OF  ASSESSES  GRADES 


Grade  N 


GO- 1  2 

1 

2 

GO- 13 

15 

22 

GS- 1 4 

44 

66 

GS- 15 

7 

10 

Totals 

67 

100 

In  all  communications  with  the  assessees  and  their 
supervisors  it  was  em-hasized  that  this  was  a  voluntary, 
research  effort  only.  It  was  made  clear  that  data  received 
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from  the  participants  would  not  be  used  in  any  personnel  de¬ 
cisions  (e.g. ,  salary  or  promotion)  and  would  not  appear  in 
any  personnel  records. 

The  overall  rating  and  dimension  ratings  from  the  as¬ 
sessment  center  (the  predictor  data)  for  the  6?  assesses:: 
were  taken  from  a  summary  sheet  of  assessment  center  results 
contained  in  each  assessee's  folder. 

The  Interview/ Questionnaire 

The  interview  and  questionnaire  (Appendix  A)  were 
constructed  sc  as  to  obtain  all  information  required  to  con¬ 
duct  an  analysis  of  the  predictive  validity  of  the  assessment 
center.  A  brief  explanation  of  some  of  the  ordering  of  the 
questions  is  warranted.  Questions  were  ordered  in  an  attempt 
to  minimise  the  cross  contamination  of  answers,  however, 
raters  were  not  prohibited  from  reviewing  any  answers  which 
they  had  given  previously. 

The  content  of  the  rating  instrument  and  the  method 
of  presentation  were  tested  in  a  trial  run  prior  to  actual 
use.  A  supervisor  in  the  civilian  personnel  division  of  ASD 
acted  as  an  assesses' s  supervisor  and  was  given  the  interview 
and  questionnaire  package.  Upon  completion  of  the  test  run, 
a  critique  of  the  rating  instrument  and  the  presentation 
method  were  discussed  and,  where  warranted,  implemented  for 

ilo ©  • 

This  rating  instrument  was  presented  to  the  raters  in 
person  whenever  possible.  This  was  accomplished  so  that  (1) 
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any  questions  or  clarifications  requested  by  the  rater  could 
be  answered  quickly,  (2)  the  fact  could  be  emphasized  that, 
this  was  a  research  effort  only,  and  (3)  the  raters  could  be 
suitably  impressed  with  the  fact  that  they  should  be  as  ob¬ 
jective  as  possible  in  their  responses.  Sven  though  the  in¬ 
terview/questionnaire  was  presented  in  oerson,  all  instruc¬ 
tions  and  questions  were  constructed  so  that  it  could  be 
completed  without  any  researcher  supervision  with  only  lim¬ 
ited  questions  and  clarifications. 

The  two  overall  ratings  (general  effectiveness  and 
effectiveness  with  respect  to  peers)  were  separated  by  a  time 
consuming  question  in  an  attempt  to  reduce  cross  contamination 
of  the  results.  In  addition,  the  scales  used  for  the  two 
ratings  were  made  dissimilar  for  the  same  purpose. 

The  rating  of  the  importance  (criticalness)  of  the  14 
dimensions  was  included  after  the  14  dimensions  were  rated  so 
as  not  to  contaminate  those  ratings.  It  was  felt  that  the 
rater  might  attempt  to  match  up  a  criticalness  rating  with 
the  dimension  rating  with  little  regard  to  the  actual  job 
usage  of  that  dimension. 

The  dimension  definitions  utilized  in  the  cuestionnair 
used  in  conjunction  with  the  dimension  rating  scales,  were 
derived  from  the  Komar  and  ’.Vise  (1980)  thesis  effort.  These 
definitions  were  modified  slightly  so  as  to  fully  consider  the 
sample  oooulation  and  integration  with  the  remainder  of  the 
interview/ questionnaire. 

Demographic  information  derived  from  the  interview 


and  not  utilized  in  this  study  is  presented  in  Appendix  3. 


The  Hat in as 

All  assecsees  received  a  rating  in  the  assessment 
censer  and  in  1931  on  14  managerial  dimensions  (5ee  Figure 
2-1).  These  dimensions  were  rated  on  a  five  point  scale. 

Three  distinct  sets  of  dimension  ratings  were  utilised  in  this 
study.  The  first  set  are  chose  derived  from  the  Komar  and 
'.Vise  study  sample  group  and  the  ratings  that  they  received  in 
the  assessment  center.  These  are  referred  to  as  the  14  assess¬ 
ment  center  dimensions.  The  second  set  are  those  dimension 
ratings  received  in  the  assessment  center  by  the  67  assessees 
samcled  in  this  study.  These  ratings  are  referred  to  as  the 
14  assessment  center  dimensions,  1981  sample  group.  Finally, 
the  third  set  are  those  dimension  ratings  given  by  the  super¬ 
visors  of  the  67  assessees  sampled  in  this  study,  referred  to 
as  the  14  1931  dimensions. 

There  are  four  overall  ratings  used  in  this  study. 

The  overall  rating  received  by  each  assesses  in  the  assess¬ 
ment  center  is  referred  to  in  this  study  as  0V3FAC  or  the 
assessment  center  overall  rating.  This  rating  is  based  on 
a  four  point  scale.  The  remaining  three  overall  ratings 
we re  ratings  derived  from  the  1931  interview/ questionnaire 
and  are  collectively  referred  to  as  the  1931  overall  ratings. 
9VF21  is  a  global  rating  of  managerial  success  using  a  five 
ooint  scale.  Due  to  a  low  number  of  *  1  *  and  '2'  ratings 
received,  these  categories  were  collapsed  into  a  single 
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category.  This  resulted  in  a  four  ocint  scale  consistent  with 
'  7T3 AG.  '  7TI?2  is  a  global  rating  of  managerial  success  in 
which  the  a ssessee  was  rated  on  how  successful  he  is  with  re¬ 
spect  to  his  peers.  This  was  measured  on  a  continuous  Ip 
centimeter  scale.  For  simplicity,  responses  were  recorded 
to  the  nearest  .5  centimeter.  The  final  overall  rating, 
7TAMDIM,  is  a  composite  of  the  14  1931  dimensions.  The 
dimensions  for  each  individual  assesses  were  summed  and  the 
resulc  divided  by  14.  Since  the  dimensions  were  rated  on  a 
five  point  scale,  this  composite  is  a  modified  five  ~cint 
scale. 


Internal  Validity  of  the  Criteria 

Overview.  The  reliability  of  the  criteria  data 
derived  from  the  supervisors  was  determined  as  follows: 

1.  examination  of  the  relations  among  the 
1981  dimensions. 

2.  examination  of  the  relations  between  the 
1931  dimensions  and  the  1931  overall 
ratings. 

3.  Identification  of  the  nature  of  the  under¬ 
lying  components  of  the  dimensions. 

4.  As  an  adjunct  to  the  first  three  steps, 
comparison  of  analysis  results  using 
1931  data,  analysis  results  using  the 
1981  assessment  center  data,  and 
analysis  results  by  Komar  and  .Vise. 
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Specific  data  analysis  techniques  used  are  summarized  in  the 
following  sections. 

Correlation  of  Dimensions.  The  correlation  among  the 
14  1921  dimension  ratings  was  accomplished  using  the  3P33 
subprogram  PEARSON  CCRR.  The  specific  objective  of  this  is 
to  insure  that  predictive  results  are  not  confounded  by  di¬ 
mensions  having  an  unexpectedly  weak  or  strong  relationship 
with  each  other. 


Dimension/Overall  Relationship.  The  correlations 
between  the  14  1981  dimensions  and  the  three  1981  overall 
ratings  was  accomplished  by  the  3F33  subprogram  PEARSON  CCRR. 
In  addition,  a  multiple  regression  analysis  to  show  the  re¬ 
lationship  between  the  14  1 93 1  dimensions  as  a  group  and  OVERT 
and  C VERS  (compared  separately)  was  accomplished  using  the 
SPSS  subprogram  REGRESSION.  This  analysis  utilized  the  step¬ 
wise  regression  ootion  in  order  to  ascertain  those  dimensions 


which  were  the  most  important  to  explaining  the  overall  ratings. 

Factor  Analysis.  The  underlying  pattern  of  relation¬ 
ships  between  the  14  1981  dimensions  was  analyzed  using  the 
SPSS  subprogram  FACTOR. 

Relationship  of  1981  Samole  Grouo  with  the  Parent 
Population.  This  analysis  serves  the  purpose  of  insuring  that 
the  underlying  perceptions  of  supervisors  concerning  job  struc¬ 
ture  have  not  changed  drastically  between  1975  and  1981.  If 
there  was  a  drastic  change  in  the  data  base,  the  analytical 
procedures  utilized  in  this  study  would  be  of  questionable 
value  and,  as  such,  a  reexamination  of  criteria  and  analysis 
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techniques  would  be  warranted. 

A  correlation  of  the  14  assessment  center  dimensions 
for  the  1981  sample  group  was  accomplished  using  the  IF 34 
subprogram  PEARSON  CORR.  This  information  was  then  visually 
compared  against  the  correlation  matrix  using  the  sample  group 
from  the  Komar  and  V/ise  (I960)  study  to  ascertain  if  the  two 
groups  could  logically,  based  on  gross  coefficient  comparisons, 
be  considered  to  be  from  the  same  overall  population. 

An  3?S3  subprogram  FACTOR  was  run  using  the  14  assess¬ 
ment  center  dimensions  from  the  1981  sample  grou-:.  The  result¬ 
ing  factor  patterns  were  compared  with  the  similar  analysis 
of  the  14  1981  dimensions  and  those  results  achieved  by  Komar 
and  '.Vise  (1980)  to  again  determine  if  they  represented  the 
same  underlying  population  of  data. 

Finally,  a  comparison  was  made  between  the  measured 
dimension  criticalness  (how  important  a  given  dimension  is 
perceived  to  be  to  managerial  success)  found  in  the  1981 
sample  group  and  that  found  by  Dr.  Hoses  in  his  initial  anal¬ 
ysis  (1973).  This  was  accomplished  using  a  Spearman's  Rho 
test  (Harnett,  1975)  on  the  rankings  of  the  dimensions.  In 
that  Dr.  Hoses'  data  was  split  between  the  A F  Avionics  Lab¬ 
oratory  (AFAL)  and  ASD,  the  1981  data  was  similarly  split 
so  as  to  give  a  more  accurate  comparison. 

Predictive  Validity 

The  SPSS  subprogram  PE ARSON  CCRR  was  accomplished  to 
determine  the  predictive  validity  of  the  assessment  center. 
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The  predictor  variable  used  was  the  assessment  center  overall 
rating,  CV3RAC.  The  criteria  of  managerial  success  used  were 
CV3S1 ,  CVE32,  and  M3AITDIM.  In  addition  to  the  comparison  of 
the  overall  ratings,  the  14  1921  dimensions  and  the  14  assess¬ 
ment  center  dimensions  (1981  sample  group  data)  were  compared 
to  each  other  and  to  the  overall  ratings  (with  the  exception 
of  5UMDIM)  using  the  SPSS  subprogram  PEARSON  CO PR. 

Subgroup  Data  Analysis 

Predictive  validity  techniques  used  for  the  overall 
sample  were  also  accomplished  on  specified  subgroups  pairs 
of  that  sample.  This  was  accomplished  to  ascertain  if  there 
were  any  significant  findings  which  could  be  attributable  to 
a  given  subgroup.  The  subgroup  pairs  analyzed  are  as  follows 

1 .  AF'.VAL  assessees  and  ASD  assessees. 

Note:  FTD  assessees  were  considered  to  be  a 
separate  group  but,  given  their  sample  size 
(N=5),  were  excluded. 

2.  Program  Managers  and  First  Level  Supervisors. 

Tote:  Other  supervisory  levels  were  not 
included  due  to  their  small  sample  size. 

3.  Military  raters  and  Civilian  raters,  l.e., 
the  assessee's  supervisor. 

4.  Assessees  with  nc  (or  inconsequential) 
budget  responsibility  and  those  with  a 
budget  responsibility. 

Selected  groups  were  also  analyzed  using  the  3? SC 
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subprogram  DISCRIMINANT.  This  program  attempts  to  statisti¬ 
cally  distinguish  between  two  groups  using  a  set  of  discrim¬ 
inating  variables  (Hie  et  al,  1975).  The  discriminating 
variables  used  are  the  14  assessment  center  dimensions  (1931 
sample  group)  and,  in  a  separate  analysis,  the  14  1981 
dimensions. 

Cunmary 

This  chapter  has  described  the  methodology  used  to 
accomplish  the  objectives.  This  included  a  review  of  prev¬ 
ious  studies  for  this  particular  assessment  center  and  a 
description  of  the  sample  used  and  how  it  was  chosen.  Finally, 
a  specific  set  of  procedures  to  achieve  the  objectives  was 
outlined.  The  next  chapter  presents  and  analyzes  the  results 
obtained  in  this  study. 
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CHAPTER  4 


ANALYSIS  AHA  RESULTS 


Cverview 

The  quantity  of  data  derived  from  the  analysis  was 
extensive.  Therefore,  only  the  collated  and  reduced  forms 
of  the  data  which  have  a  significant  impact  on  the  explanation 
of  the  objectives  are  presented  here.  In  addition,  data  from 
the  Komar  and  '.Vise  (I960)  research  is  used  extensively. 

Internal  Validity  of  the  Criteria 

The  criteria  data  were  analysed  to  determine  (1)  if 
the  sample  population  could  be  considered  to  be  a  valid  sample 
of  the  overall  assessment  center  population  and  (2)  if  the 
responses  given  by  raters  could  be  considered  to  be  reliable. 

Correlation  of  Dimensions.  The  correlation  coeffi¬ 
cients  among  the  14  assessment  dimensions  (1981  sample  pop¬ 
ulation)  are  shown  in  Table  4-1.  Table  4-2  shows  the  correl¬ 
ation  coefficients  found  by  Komar  and  ’.Vise  (I960)  for  the  14 
assessment  center  dimensions.  Finally,  Table  1-3  reflects 
the  correlation  coefficients  for  the  14  1981  supervisory 
dimension  ratings  in  addition  to  the  1931  overall  ratings. 

Assessment  Center  Relationship.  Table  4-1  reflects 
the  1981  sample  assessment  center  dimension  correlations  as 
well  as  their  correlations  to  the  overall  assessment  rating. 
This  is  as  compared  to  the  Komar  and  .Vise  results,  Table  4-2. 
As  with  Plomar  and  Vise  all  results  were  highly  significant 
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TABLE  4-2 

CORRELATIONS  AMONG  DIMENSION  RATINGS  (KOMAR  AND  WISE) 

(N=144) 
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TABLE  4-3 

CORRELATIONS  AMONG  1981  OVERALL  AND  DIMENSION  RATING 

(N=67) 


ee  Figure  2-1  for  explanation  of  dimension 
significance:  p<.05  except  as  noted 


(at  greater  than  the  .05  level)  except  those  associated  with 
the  Apathy  dimension.  Many  of  these  coefficients  were  sig¬ 
nificant  at  the  .10  level  and  some  were  not  significant. 

A  visual  comparison  between  Table  4-1  and  Table  4-2 
reveals  that  the  1931  sample  compares  favorably  to  those 
results  achieved  by  Komar  and  7/ise.  This  lends  credence  to 
the  necessary  condition  that  the  1961  sample  be  a  valid  sub¬ 
set  of  the  Komar  and  7/ise  sample  population  and  the  parent 
oo  oul ation. 

The  strongest  correlations  between  the  overall  assess¬ 
ment  center  rating  and  dimensions  on  Table  4-1  (for  the  1981 
s am ole)  were  Leadership  (.82),  Flexibility  (.80),  Persuasive¬ 
ness  (.79),  Forcefulness  (.76),  Organising  Okills  (.76),  and 
Decision  Making  (.74).  The  weakest  correlations  were  Written 
Communications  Skills  (.46)  and  Empathy  (.20).  These  findings 
relate  ’.veil  to  the  Komar  and  7/ise  study  which  found  Persuasive¬ 
ness,  Leadership,  Decision  Making,  Problem  Solving,  Organizing, 
and  Forcefulness  as  the  most  highly  correlated  with  the  overall 
assessment  rating  and  7/ritten  Communications  Skills  and  Em¬ 
pathy  as  the  lowest  correlated  (Komar  and  7/ise,  1980). 

Dimensions/Overall  Fating  Relationship.  The  overall 
rating  using  the  absolute  scale  (07FP1)  and  the  overall  rating 
comparing  the  assessee  with  his  peers  (CVED2)  were  compared 
to  each  other  and  to  the  14  1981  supervisory  dimension  ratings. 
The  resultant  correlation  coefficients  are  shown  in  Table  4-5. 
All  coefficients  were  significant  at  greater  than  the  .05 
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level  except  Problem  Solving  v/ith  Snoathy  and  Organizing 
Skills  with  Flexibility.  These  were  marginally  significant 
at  between  the  .05  and  the  .10  level. 

A  further  review  indicates  that  the  strongest  cor¬ 
relations  between  the  supervisory  ratings  and  the  CVFR1 
overall  rating  were  Decision  Making  (.61),  Problem  Solving 
(.61),  and  Leadership  (.59).  Snergy  (.30)  had  the  lowest 
correlation.  The  strongest  correlation  between  the  surrvi- 
scry  ratings  and  the  CVFR2  overall  rating  we re,  again,  Prob¬ 
lem  Solving  (.75) »  Leadership  (.68),  and  Decision  Making 
(.66),  but  additionally  included  Forcefulness  (.66).  The 
weakest  correlations  were  Tmpathy  (.40)  and  '.Yritten  Com¬ 
munications  Skills  (.40).  These  findings  again  relate  well 
to  the  results  found  in  the  Komar  and  ’.Vise  study. 

In  order  to  determine  which  of  the  14  1981  dimensions, 

as  a  group,  best  related  to  the  overall  ratings,  multiple 

regression  analysis  was  accomplished.  Variables  were  entered 

into  the  equation  in  a  stepwise  fashion.  As  with  Komar  and 

2 

'.Vise  (1930)  some  variable  subsets  resulted  in  a  high  R  value 
and  are  included  in  the  final  analysis. 

The  multiple  regression  equation  involving  the  14 
1931  dimensions  and  CVLR1 ,  the  absolute  overall  rating,  is 
shewn  in  Table  4-4.  Inclusion  of  additional  variables  from 
those  shown  resulted  in  an  R^  increase  no  greater  than  .02. 

The  variables  included,  Decision  Malting,  Flexibility,  Organ¬ 
izing  Skills,  and  Oral  Communications  Skills,  account  for 

5o.9r/o  of  the  variance  of  CVER1 .  Decision  Making  has  the 
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highest  simple  correlation  of  all  the  dimensions. 

TA3L3  4-4 

MULT  I  I'Ll  R33333SICN  OF  1 9  ~  1  DIMFUoICIT  FATIMC-,: 
CM  CVTP1  RATING 


DU 
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.279 

37.97 

.000 

TTT 
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.485 

.  109 

.190 

29.21 

.000 

C3 

.534 

.049 

.185 

23.26 

.000 

1  CCS 

.569 

.035 

.  1 66 

19.82 

.000 

1 

|  Constant 

-.219 

The  multiple  regression  equation  involving  the  11  1951 
dimensions  and  0Y3R2,  the  overall  rating  with  respect  to 
peers,  is  shown  in  Table  4-5.  Any  subsequent  inclusion  of 
variables  would  result  in  an  increase  no  greater  than 
.009.  The  variables,  Problem  Solving,  Leadership,  Forceful - 
ness,  and  Stress  Tolerance,  explain  70.5fj  of  the  variance  of 
C7TR2.  Problem  Solving,  Leadership  and  Forcefulness  had  three 
of  the  highest  simple  correlations  with  CVFR2. 

As  a  measure  of  comparison  between  the  assessment 
center  (1981  sample  group)  and  the  1981  dimensions,  Table 
4-6  reflects  the  means  and  standard  deviations  of  these 
variables.  Both  sample  groups  were  measured  on  a  five  point 
scale,  one  being  the  lowest  value  and  five  the  highest  val¬ 
ue  for  effectiveness  in  that  dimension.  Table  4-7  reflects 
the  ranking  of  the  dimension  means  and  their  relative 
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ordering  with  respect  to  each  ether. 


TABLE  4-5 

ITJLTIPLE  REGR333ICIJ  C?  1931  OlnEITCICi:  RATINGC 
ON  CV2R2  RATING 


Vari  a: 

;le  32 

2 

3  Increase 

3  '.’/eight 

Overall  ? 

Dig  of  ? 

i  o 

.556 

— 

1 .071 

79.00 

.000 

L 

.666 

.110 

.555 

61 .32 

.000 

?0 

.684 

.013 

.h-36 

44.10 

.000 

:)  - 

.705 

.021 

.451 

55.32 

.000 

Const: 

ant 

.059 

In  all  cases  except  Cral  Communications  Skills  and 
Stress  Tolerance,  the  1931  dimension  means  were  greater  in 
magnitude  than  the  assessment  dimension  means.  In  addition, 
most  of  the  assessment  dimension  standard  deviations  were 
greater  than  the  1931  dimension  standard  deviations.  This 
implies  that  the  1931  raters  were  more  prone  to  give  high 

J 

ratings  with  less  variance  from  that  rating.  This  suggests 
either  the  possibility  of  rating  inflation  on  the  raters 
part,  a  strictness  in  the  assessment  center  ratings,  or  the 
possibility  that  all  assessees  truly  improved  themselves  in 
all  dimensions.  Turther  research  on  these  findings  may  be 
warranted. 


Dimension  Criticalness  Ratings, 
tain  if  there  was  any  significant  shift 
importance  of  the  dimensions,  the  means 


In  order  to  asesr- 
in  the  perceived 
for  the  assessment 
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TABLE  4-6 
VIATICIIE  0? 


::exci:e  (1931  ealelz) 
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Means/ ( Standard  Eeviations) 

Bee  Figure  2-1  for  explanation  of  dimensions 
11=67  (Mote:  11=66  for  asterisked  dimensions) 
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TABLE  4-7 


HANKING  CF  DIMENSION  MEANS  (1931  SAMPLE) 
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See  Figure  2-1  for  explanation  of  dimensions 


S' ear man' s  Fho 


r  2  Sig  (t-test) 

o  o 

.24  .06  .10 
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An  examination  of  the  means  reveals  that,  in  general, 
there  were  no  major  differences  between  the  two  sample  groups. 
Exceptions  to  this  are  a  fairly  large  (approximately  15;'$  or 
0.5  on  a  five  point  scale)  decrease  in  importance  of  the  Risk 
Taking  dimension  and  a  large  increase  in  importance  of  the 
Acquiring  Information  dimension  within  both  subgroups  and 
of  the  Energy  dimension  within  the  AF'.’/AL  subgroup.  One 
possible  reason  for  this  change  is  that  perhaps  management 
emphasis  is  changing  from  a  deductive,  or  even  luck,  basis 
(Risk  Taking)  to  an  inductive  basis  (Acquiring  Information), 
The  dimensions  were  ranked  (Table  4-9)  according  to 
the  magnitude  of  their  means.  Ties  were  broken  by  a  compari¬ 
son  of  the  modes  of  the  dimensions.  A  rank  order  correlation 


also  holds 


Assessment  Center 
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AFAL  AST)  AF.7AL 

Dimensions  (;T=87)  (N=60)  (N=25) 
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The  only  notable  exception  to  this  is  in  the  comparison  of 
the  1973  and  1931  AS!  ranking  of  Problem  Solving,  which  rose 
from  tenth  o lace  to  second.  Ko  explanation  for  this  is 


apparent . 

factor  Analysis.  A  factor  analysis  (Varimax  rotation 
with  Kaiser  normalization,  as  with  Komar  and  '.Vise)  of  both 
sets  of  14  dimensions  (assessment  center  ratings  with  the  1981 
sample  and  the  1931  ratings)  was  accomplished  to  identify  any 
underlying  factors  and  to  compare  the  Komar  and  '.Vise  factor 
analysis,  Table  4-10. 

The  analysis  of  the  14  assessment  center  ratings 
(1921  sample  group)  resulted  in  the  identification  of  three 
factors  with  eigenvalues  of  7.71,  1.19,  and  1.07.  Results 
are  shown  on  Table  4-11.  Factor  I  was  comprised  of  Leadership, 
energy,  Forcefulness,  Persuasiveness,  and  Flexibility.  This 
equates  to  the  first  factor  of  Komar  and  'Vise  and  is  referred 
to  as  interpersonal  skills.  Factor  II  includes  Pisk  Talcing, 
Acquiring  Information,  Organizing  Skills,  Problem  Solving, 
and  Decision  Making.  This  is  again  consistent  to  the  second 
factor  in  Komar  and  Wise  and  may  be  referred  to  as  adminis¬ 
trative  skills.  The  third  factor  was  determined  by  Oral 
Communications  Skills.  Stress  Tolerance  contributed  to  fac¬ 
tors  I  and  III  while  Written  Communications  Skills  contributed 
to  factors  II  and  III.  Zmpathy  made  no  significant  contribu¬ 
tion  to  any  of  the  factors. 

Table  4-12  reflects  the  factor  analysis  of  the  id 
1921  dimensions.  The  analysis  originally  Listed  two  factors 


TABLE  4- 1 1 


FACTOR 

1LY3IS  SEOUL 

TIC  • 
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Figure  2-1 

for 

explanation  of  dimers 

ions 
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TABLE  4-12 

ALY3I.3  RESULTS:  LCADI27G  AFTER  VARIMAX  ROTATI' 
SUPERVISORY  TATA,  1931  3 APPLE 
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See  Figure  2-1  for  explanation  of  dimensions 
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as  having  eigenvalues  greater  than  one.  In  or her  to  be  c 
sistent  with  the  other  factor  analyses,  the  cutout  was  force! 
to  three  factors  with  eigenvalues  of  6.62,  1.32,  and  0.2-8. 
Factor  I  consists  of  Forcefulness,  Acquiring  Information, 
Organising  Skills,  Problem  .Solving,  and  Decision  Faking. 

This  compares  favorably  to  the  second  factor  in  the  other 
analyses.  Factor  II  consists  of  Leadership,  Empathy,  Persua¬ 


siveness,  Flexibility,  and  Stress  Tolerance.  This  compares 


favorably  to  the  first  factor  of  the  analyses  using  assess¬ 
ment  center  dimension  data.  The  third  factor  includes  Energy 
and  Risk  Taking.  Oral  Communications  Skills  and  ",'ritten  Com¬ 
munications  ’kills  contribute  to  both  the  first  and  second 
factors. 


Predictive  Falldity 

The  predictive  validity  consisted  of  a  computation  of 
the  correlation  coefficients  of  the  overall  assessment  center 
rating  and  the  14  assessment  center  dimension  ratings  with 
the  14  1931  dimension  ratings  and  the  two  overall  ratings 
used  as  criteria  measuring  managerial  success.  In  addition, 
stepwise  multiple  regression  analyses  v/ere  accomplished  be¬ 
tween  the  overall  assessment  center  ratings  and  between  the 
14  assessment  center  dimensions  and  the  two  1?3l  overall 
ratings. 

Table  4-13  reflects  a  matrix  of  correlation  coeffi¬ 
cients  of  all  involved  variables.  The  overall  ratings  for 
1981  had  very  low  correlation  coefficients  with  the  assess¬ 
ment  center  overall  rating  (.002  -with  C7EE1  and  -.019  with  C7EE2). 
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:  11=66 

ee  Figure  2-1  for  explanation  of  dimension 
ignificance:  p>.05  except  as  noted 

*.01  <  .05 

**p£.01 


The  significance  of  these  correlations  greatly  e;:ceeued  the 
.35  level  end,  in  fact,  can  be  considered  chance  occurences. 

The  only  1981  dimensions  which  shewed  fairly  consis¬ 
tent  correlations  with  the  assessment  center  data  were  ~ral 
Communications  Chills  and  Energy.  In  a  like  manner,  the 
assessment  center  dimensions  Oral  Communications  Skills  and 
.'.’rittsn  Communications  Skills  correlated  significantly  (at 
less  than  the  .05  level)  with  many  of  the  1981  dimension 
and  overall  ratings. 

Overall,  the  predictive  validity  results  in  this 
research  effort  compare  very  poorly  with  the  Komar  and  '.'.'iso 
results  (shown  on  Table  2-3)  and  with  those  results  shown 
on  Table  2-4. 

Multiple  regressions  between  the  1931  supervisory 
ratings  and  assessment  center  data  were  run  to  ascertain  if 
any  predictive  validity  may  be  realised.  Table  4-14  reflects 


the  multiple  regression  of  the  Ik  assessment 


>nter  aimensirn 


ratings  end  the  C7T21  rating.  The  inclusion  of  additional 

variables  beyond  those  listed  resulted  in  no  greater  than  a 

o 

.02  increase  in  2“  for  any  one  variable.  In  addition,  inclu¬ 
sion  of  additional  variables  caused  a  further  inflation  of 
the  significance  (i.e.,  a  degradation)  of  the  equation,  there¬ 
fore,  analysis  was  terminated. 

Table  4- 1 5  reflects  the  multiple  regression  of  the 


14  assessment  center  dimension  ratings  and  the  1981  P’^OO 
rating.  As  can  be  seen,  although  statistically  significant, 
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the  four  variables  included  (  Vritten  Communications  Skills, 
Empathy,  Zorcefulness,  and  Cral  Communications  Ckills)  only 
-account  for  21  .7'3  of  the  variation  of  the  overall  rating  uced . 
Tha  inclusion  of  additional  variables  (beyond  these  four) 
in  the  equation  did  not  add  to  the  2“  value  aero  than  .OOd 
for  any  one  variable.  Empathy  and  Forcefulness  correlated 
■vith  the  0 7:322  rating  negativally  (thus,  the  negative  lets 
■.veights)  and  '.Vritten  Co mmuni cations  Ckills  had  the  highest 
single  single  correlation  -.vith  the  C7ZE2  rating  (Table  1— 15). 

3  0  cuati  on  to  ^rsssnbs  ths  "b  -3  s  b  72r0d.icti.v0  r*  0  3  u  i  t  c  of  this 
study  (for  ACT  and  the  labs  combined). 

TABLT  A- 1 A 


-_i  .  « _ 


IXLTITLZ  2232Zo3IC:r  C?  ACCZCCMZZT 

iisici:  rati ::cs  (1951  oamplz)  c:r  cvzzi  zatizg 


Variable 

:> 

3*“  Increase 

3  Weight 

Overall  Z 

Cig  of  ' 

- 

r\x 

— 

.125 

1 .94 

*  r 

.  i  oy 

Constant 

2 » b  o  a 

The  multiple  regression  between  the  Id  1?21  dimen¬ 
sions  and  the  overall  assessment  center  rating  (C7Z2AC)  is 
shown  in  Table  A- 1 6 .  The  four  variables  included  (Tral  Com¬ 
munications  Skills,  Empathy,  Znergy,  and  Zorcefulness)  account 
for  only  20'5  of  the  variance  of  07Z2AC,  but  are  statistically 
significant.  Inclusion  of  additional  variables  beyond  these 
four  did  not  increase  the  2^  value  any  more  than  ,0A  for  any 
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single  variable.  ral  Conaunic 
the  highest  single  correlations 
of  using  one  gredic  tor  variable 
variables  is  liaised. 


ations  Skills  and  Energy  ha 


with  CV3RAC.  She  usefulne 
so  exglain  four  criteria 


TABLE  4-15 

MULTIPLE  RZCRESSICP 

r-  *71  ^ 

j.  nO‘J_i  _✓  •«;  l’lHi  <  ^ 

cutter  oikeitsio 

RATIkGo  (1931 

O.s.1  . .  o_i  _j  y  — i  v  > 

1R2  RATI PC 

2 

Variable  3 

p2 

increase  2  '.’/eight 

Overall  7 

r  o  ■** 
o  — 

'.700  .074 

.917 

.029 

1  -o 

-j-  -  •  ■  >  ✓ 

.063  -.935 

4.99 

.010 

FT  .  1 34 

.045  -.730 

1.53 

.006 

CCS  .217 

.055  .706 

4.17 

.005 

Constant 

9.869 

TABLE  4-16 

t  *trf 

.  -UlJ 

71  PI 

.2  RUGRZSC Id  CP  1931 

0IPZP3IC II 

RAZIPGS  CP  OVZRAC  RA 

TIITC- 

Variable  FT 

Increase  3  '.'/eight 

Overall  7 

Gig  of 

CCS  .09 

_  ."OR 

6.51 

.015 

UK  .  1 3 

.04  -.212 

4.49 

.015 

e:t  .  1 5 

.02  .514 

^  ^  f 

.017 

?C  .20 

.05  -.266 

5.67 

.010 

Constant 

1.305 

final  criterion  of  ranagerial  effectiveness  used 
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,uw/  .-iij,;  • 


cst  significant  results  cf  this  study 


(;:=4c )  resulted  in  insignificant  findings.  The  ccefficien 
was  .16  with  a  significance  cf  .134. 

occurred  between  the  subgroup  of  A?' 7.11  assessees  (1=37).  A 
correlation  analysis  between  C7HDA C,  the  overall  assessment 
center  rating,  and  CVHI?2,  the  overall  rating  with  respect  to 
peers,  -was  accomplished  in  each  cf  the  organisation  subgroups, 
The  A77/AL  correlation  coefficient  was  .37  for  a  significance 
of  .035.  This  implies  that  within  the  A77/AL  laboratory  coa- 
olen,  the  assessment  center  is  a  valid  predictor  of  managerial, 
effectiveness. 

rnV}g  correlation  coofiicisni  *00  two  on  C*/IjI?AC  cind 
within  ASD  organizations  was  again  significant  at  the  .043 
level.  However,  the  coefficient,  -.24,  implies  that  the 
assessment  center  predicts  the  opposite  managerial  effective¬ 
ness  from  chat  which  the  assesses  ultimately  achieves  (based 
on  the  current  supervisor's  rating).  This  is  net  \  nlike  those 
results  found  with  the  military  rater  subgrou".  A  conpariocn 
of  Fisher's  Z  test  (Guilford  and  Fruchter,  1?73)  revealed 
that  these  groups  are  significantly  different  at  less  than 
the  .02  level. 


The 

correlatio 

n  between  C7HRAC  and  the  abaci 

ute  1 

overall 

rating,  CVF21 , 

was  insignificant  for  both  AF 

r.7AL 

(r=,22, 

si  2= 

.146)  and 

AID  (r=-.22,  sig=.0?2).  Cnee 

again 

though, 

the 

AGO  value 

•.•/as  negative  implying  reverse 

.redi 

;r  the  assessment  center  overall  predictor  variable. 
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In  that  these  overall 


predictive  validity  results 
•."■ore  confounded  by  the  extreme  ratings  in  opposite  directions 
a  discriminant  analysis  was  acccmolished  using  AF'.VAL  and  All 
as  the  groups  to  be  distinguished.  The  first  analysis  was 
done  using  the  14  assessment  center  dimensions.  Results  of 
this  analysis  are  shewn  in  Table  4-17.  Persuasiveness,  Risk 
Talcing,  and  A’ritton  Communications  Skills  were  the  variables 
entered  which  had  the  best  discriminative  powers.  Persuasive 
ness  was  more  important  to  identifying  All  while  Pisk  Taking 


and  Written  Communications  Skills  were  important  to  identwiyi 
A7T.IL.  liven  the  significance  of  the  discriminant  function 
(.076)  and  Bex’s  I!  equality  test  (.002),  there  was  no  signif¬ 
icant  difference  between  the  two  groups. 

Table  4-1 8  shows  the  results  of  analysis  using  the 
1931  dimension  ratings.  Dimensions  considered  significant 
were  Energy,  Empathy,  Flexibility,  and  Problem  Solving, 
Dimensions  important  to  the  identification  of  AF’VAL  were 
Energy  and  Flexibility.  Dimensions  important  to  identifi¬ 
cation  of  A3D  were  Empathy  and  Problem  Solving.  The  sig¬ 
nificance  of  the  discriminant  function  (.0007)  and  Box's 
V-  equality  test  (.084)  leads  to  the  conclusion  that  these 
two  groups  are  significantly  different. 


Summary 

An  analysis  of  the  data  resulting  from  the  interview/ 
questionnaire  package  was  accomplished.  The  relationshi  - 
of  the  data  to  the  Komar  and  wise  and  assessment  center 
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TA3LB 

A- 17 

discniminant 

ANALYSIS  USING 

A332S3I  12I.T 

Tr-»^T'-r»7'r>  -XT'  't-y  t 

4/  -j*. _ _  i\  i'w  .  ,  > 

Classif 

Nunc 

ication 

tions 

Biscriainant 

Junction 

T  T  0 

AJ'VAL 

ASO 

—  ■£■> 

1.31 

1  .72 

.515 

->,71 

3.02 

2.45 

-.705 

■  >t~t  r% 

.4  V^.  7 

3.74 

3. 13 

-.733 

il  o  u  c£Il  u 

-15.27 

-12.47 

Signific 

ance 

Di 

scrininant  Nunc 

tion 

.075 

Bo 

x's  M  2 quality 

Test 

.002 

2A3L2 

4-13 

DI3C2IHINAN?  ANALYSIS 

USING  1931 

DINBNSICNS 

’r£_V'‘?  ro  n]_  q 

Classification 

Junctions 

AJ'.'/AL  A  SB 

Oiscriainant 

Junction 

TT 

5.53 

4.21 

-.926 

r 

2.52 

3.53 

.373 

JL 

1.57 

1.09 

-.429 

-J  C 

-  V 

1.40 

3.25 

•  o  y-j 

Constant 

-25.50 

-25.75 

Significance 

Biscriainant  Junction 
Loir's  I!  Bcuality  Test 
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"■ovulation  was  determined  and  the  internal  validity  of 
data  v.-as  examined.  The  predictive  validity  of  the  asse 
ment  center  using  interview/ questionnaire  criteria  v/as 
.wined.  Finally,  an  analysis  of  predictive  validity  for 
.4.33  and  A7TAL  subgroups  v; as  accomplished.  The  final  ch 
of  this  study  presents  conclusions  and  recommendations 

this  and  prior  chapters. 


t  h  o 


er 


based 


i 


on  the 


results  of 


CHAP?’?*?  5 


HJCLU3ICN3  AHD  RECO M-IEITT ATI C N 


nummary 

Assessment  centers  have  been  used  successfully  in 
government  and  industry.  Success  in  a  given  environment  is 
dependent  on  the  ability  of  an  assessment  center  to  accurat¬ 
ely  predict  that  which  it  is  designed  to  predict.  It  must 
have  predictive  validity. 

The  1974-1975  assessment  center  at  AST  was  designed 
to  credict  managerial  effectiveness  in  supervisory  ’"•ositions 
in  AST  and  AE'.VAL,  A  study  by  Komar  and  ’.Vise  utilized  a  sub¬ 
set  of  the  assessment  center  population  and  concluded,  based 
on  analysis  of  certain  criteria,  that  the  assessment  center 
does  have  predictive  validity,  albeit  at  a  moderate  level. 

This  research  also  studied  a  subset  of  the  1974-1975 
assessment  center  population  in  an  attempt  to  better  define 
the  predictive  validity.  Alternate  criteria  of  managerial 
effectiveness  were  used  in  this  study.  A  subgroup  analysis 
was  included  based  on  the  organization  to  which  the  67  assess- 
ees  belonged  (either  AE’.VAL  or  ATT), 


Conclusions 

Based  on  an  analysis  of  criteria  for  managerial  success 
and  a  study  of  the  work  environment,  the  criteria  selected  in 
this  study  appear  valid. 

The  results  indicate  that  the  sample  of  this  study 
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the 


is  very  similar  to  the  Komar  and  '.Vise  sample.  Also, 
comparison  of  the  underlying  dimensionality  (factor  analysis) 
of  the  1931  supervisory  ratings  and  the  1975  assessors' 
ratings  show  very  similar  patterns  for  conceptualizing  the 
work  environment.  The  results  of  this  study,  therefore, 
should  compare  favorably  with  the  Komar  and  '.Vise  study. 

Given  valid  criteria  and  a  representative  sample, 
the  results  of  this  study  do  not  support  the  predictive 
validity  of  the  1974-1975  assessment  center.  The  correla¬ 
tions  between  the  assessment  center  rating  and  the  two  1931 
overall  effectiveness  ratings  were  not  significantly  differ¬ 
ent  than  zero.  Results  indicate,  however,  that  if  its  use 
was  limited  to  the  prediction  of  managerial  effectiveness 
in  AF'.'/AL,  then  the  assessment  does  have  predictive  validity 
at  a  greater  level  than  that  indicated  by  Komar  and  Vise, 

ITo  apparent  reason  is  evident  as  to  the  reason  the  ass“s<f^ 
ment  center  oredicts  success  in  the  AF'.’/AL  environment  and  net 
in  the  A3D  environment. 

Several  possible  overall  conclusions  are  offered: 

1.  If  this  study  —  criteria,  sample,  analysis, 
methodology  —  is  accepted,  then  the  Komar 
and  '.Vise  criteria  must  be  judged  invalid 
and  the  predictive  validity  of  the  assess¬ 
ment  center  left  undemonstrated, 

2.  Both  sets  of  criteria  may  be  accepted  and  the 
conclusion  drawn  that  the  assessment  center 
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possesses  predictive  validity  only  for  the 
AF7/AL  job  environment. 

3.  This  study,  specifically  the  criteria  used, 
may  be  judged  or  demonstrated  to  be  flawed. 

Any  significant  results  (positive  or  neg¬ 
ative)  found  here  might  be  considered  to 
have  been  achieved  randomly.  The  assess¬ 
ment  center  then  would  continue  to  be  judged 
as  having  predictive  validity. 

As  a  final  conclusion,  in  that  comparisons  of  result 
within  this  study  and  between  Komar  and  Wise  results  and  thi 
study  are  marginally  conclusive  and  confusing,  further  re¬ 
search  appears  to  be,  not  only  apparent ,  but  necessary. 

Recommendations 

Given  the  disparity  between  results  of  this  research 
effort  and  that  of  Komar  and  7/ise  it  is  apparent  that  furthe 
study  of  the  assessment  center  and  criteria  for  managerial 
success  is  warranted.  The  following  recommendations  may  be 
considered  for  further  investigation: 

1.  Investigate  the  validity  of  the  overall 
rating  criteria  used  in  this  study  through 
the  use  of  a  comparison  with  merit  appraisal 
ratings. 

2.  Interviews  with  supervisor-raters  who  were 
also  assessees  revealed  that  it  was  possible 
that  information  on  how  to  do  well  in,  or 
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game,  the  assessment  center  was  passed 
on  from  past  assessees  to  those  slated 
to  go.  A  time  trend  analysis  of  assess¬ 
ment  center  results  (both  overall  and  the 
individual  dimensions)  may  be  investi¬ 
gated  to  ascertain  if  rating  inflation 
exists. 


3.  Another  criterion  which  may  be  utilised 
to  test  the  validity  of  the  assessment 
center  is  the  time  interval  in  promo¬ 
tions  of  assessees,  i.e.,  time  to  promo¬ 
tion  from  the  completion  of  the  assess¬ 
ment  center  or  time  between  subsequent 
promotions.  This  is  as  differentiated 
from  the  Komar  and  '.Vise  study  which  used 
only  the  number  of  promotions  as  a  cri¬ 
terion. 

4.  The  validity  of  criteria  may  be  tested  in 

a  comparison  of  results  using  the  Komar  and 
'.Vise  criteria  and  those  results  using  criteria 
from  this  study.  This  should  be  limited  to 
the  67  assessees  used  in  this  study  in  that 
data  is  most  readily  available  for  these 
subjects. 

5.  Given  the  readily  apparent  difference  in 
predictive  validity  between  A.3D  and  AF'VAl 
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in  this  study,  a  similar  analysis  of  Komar 
and  '.Vise  data  is  warranted.  An  analysis  of 
the  four  Komar  and  '.Vise  criteria  within  A  .3D 
and  AK'.VAL  should  be  accomplished  and  the 
results  compared  to  these  results  found  in 
this  study. 
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good  and  sore  not  so  good  characteristics  v;ho  clearly  does  an 
acceptable  job. 

'■•'IPIMiLLY  3TT PPP.loPPL :  Is  an  acceptable,  but  scrietir.es  mar¬ 
ginal  supervisor/manager .  Phis  person  and,  due  to  his  super¬ 
vision,  those  r/o rhino  for  him  at  tires  do  very  good  work  but 
there  are  scretires  serious  weaknesses.  Phis  person  has  sore 
good  skill s/knovl edges  but  needs  to  work  on  sore  important 
areas.  This  person  might  be  considered  a  suoervisor/manager 
who  requires  monitoring  and  occasional  help  by  higher  manage¬ 
ment  but  who  can  do  acceptable  work  with  such  hel :. 

"!I S IT C 3P3 3 7UL :  Is  a  marginal  performer  as  a  supervisor/man- 
ager.  Phis  person  and,  due  to  his  supervision,  those  working 
for  him  are  often  a  source  of  concern  for  higher  management. 
Acceptable  work  is  at  times  produced,  but  this  oerson  is 
clearly  lacking  in  some  important  skill/knowledge  areas.  Phis 
person  is  a  superviscr/manager  about  which  some  action  will 
probably  be  taken — transfer,  training,  special  assignments, 
or  others. 
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Briefly  describe/highlight  an  incident  in  which  the  subject 
was  most  effective  in  his  present  job: 


Briefly  describe/highlight  an  incident  in  which  the  subject 
was  least  effective  in  his  present  job: 
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Compared  to  his  peers,  how  would  you  rate  him  with  respect 
to  overall  effectiveness  on  the  following  scale? 


FOR  RESEARCH  USE  ONLY 
Circle  Appropriate  Response 


Oral  Communications  Skills 


1 

1 

1 

1 

1 

Poor 

Fair 

Good 

Excellent 

Outstanding 

Leadership 

1 

1 

1 

1 

1 

Poor 

Fair 

Good 

Excellent 

Outstanding 

Energy 

1 

1 

1 

1 

1 

Poor 

Fair 

Good 

Exc ellent 

Outstanding 

Forcefulness 

1 

1 

1 

1 

1 

Poor 

Fair 

Good 

Excellent 

Outstanding 

Persuasiveness 

1 

1 

1 

1 

1 

Poor 

Fair 

Good 

Excellent 

Outstanding 
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Circle  Appropriate  Response 

Flexibility 


1 

1 

1 

1 

1 

Poor 

Fair 

Good 

Excellent 

Outstanding 

Smoathy 

1 

1 

1 

1 

1 

Poor 

Fair 

Good 

Excellent 

Outstanding 

Stress  Tolerance 

1 

1 

1 

1 

1 

Poor 

Fair 

Good 

Exc ellent 

Outstanding 

Risk  Taking 

1 

1 

1 

1 

1 

Poor 

Fair 

Good 

Excellent 

Outstanding 

Acquiring  Information 

1 

1 

1 

1 

1 

Poor 

Fair 

Good 

Excellent 

Outstanding 
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FOR  RESEARCH  USE  ONLY 
Circle  Appropriate  Response 


Organizing  Skills 


1 

1 

1 

1 

1 

Poor 

Fair 

Good 

Excellent 

Outstanding 

Problem  Solving 

1 

1 

1 

1 

1 

Poor 

Fair 

Good 

Excellent 

Outstanding 

Decision  Making 

1  1 

1 

1 

1 

Poor 

Fair 

Good 

Excellent 

Outstanding 

Written  Communications 

Skills 

1 

1 

1 

1 

1 

Poor 

Fair 

Good 

Excellent 

Outstanding 
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ORAL  COMMUNICATION  SKILLS:  How  effective  is  this  individual 
in  expressing  ideas  orally?  Are  presentations  logical  and 
well  organized? 

Additional  points  to  consider: 

Volume  and  projection 

Voice  clarity  and  articulation 

Voice  tone-pleasant  and  enthusiastic  or  monotone? 

Eye  contact,  gestures,  and  facial  expressions 
Use  of  notes 
Distracting  habits 
Organization 

Note;.  Vocabulary  and  formal  correctness  are  not  critical 
unless  they  distract  from  the  message  being  communicated. 

LEADERSHIP:  To  what  extent  is  this  individual  able  to  get 
others  to  work  together  as  a  group  in  accomplishing  a  task? 

Additional  points  to  consider: 

Attempts  to  lead.  It  is  better  to  attempt  to  lead 
and  fail  than  to  never  even  attempt  to  lead. 
Effectiveness  in  coping  with  dissension  and  bring¬ 
ing  about  compromises. 

Appropriateness  of  leadership  style. 

Success  in  gaining  group  support. 

ENERGY:  To  what  extent  does  this  individual  maintain  a 
high  level  of  activity?  How  much  does  this  individual 
participate  in  activities? 

Additional  points  to  consider: 

Verbal  inputs 
Note  taking 

Consider  the  level  of  activity.  Do  not  be  concerned 
with  the  impact  of  the  activity  on  the  group. 
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FORCEFUL NESS:  To  what  extent  is  this  individual  able  to 
command  attention  from  others?  Is  this  individual's  pre¬ 
sence  felt  by  others?  Is  there  any  attempt  to  actively  seek 
attention  rather  than  passively  react  to  events  as  they  occur? 

Additional  points  to  consider: 

Aggressiveness 

Gaining  attention  and  reaction 
Self  confidence 
Display  of  initiative 

Note:  The  difference  between  Leadership  and  Forcefulness 

is  that  while  an  individual  may  be  forceful,  i.e.,  able  to 
command  attention,  this  individual  may  not  be  effective  in 
leading  others  to  accomplish  a  task. 

PERSUASIVENESS :  To  what  extent  is  this  individual  able  to 
sell  his/her  point  of  view  and  counter  objections  from 
others? 

Additional  points  to  consider: 

Attempts  to  sell  ideas.  The  fact  that  a  person  trys 
is  worth  more  than  if  they  did  not  even  try. 

Success  in  selling  ideas 
Is  this  individual  convincing? 

Note:  The  difference  between  Leadership  and  Persuasiveness 
is  that  an  individual  may  be  able  to  sell  ideas  to  others, 
but  ineffective  in  bringing  others  together  to  accomplish 
the  whole  task. 
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FLEXIBILITY:  To  what  extent  can  this  individual  adjust  to 
new  situations  easily? 

Additional  points  to  consider: 

Reaction  to  new  information 

Variation  in  behavior  and  style  with  different  people 
Willingness  to  compromise 
Variation  in  tactics 
Acceptance  of  other's  ideas 

Ability  to  change  roles  (e.g.,  from  leader  to  follower) 

EMPATHY :  To  what  extent  does  this  individual  show  concern 
for  others? 

Additional  points  to  consider: 

Sensitivity  or  awareness  of  others'  feelings 

Respect  while  others  are  speaking 

Do  others  feel  comfortable  around  this  individual? 

STRESS  TOLERANCE:  To  what  extent  does  this  individual  exhibit 
stability  of  performance  under  pressure,  opposition,  or  other 
difficult  conditions? 

Additional  points  to  consider: 

Poise 

Display  of  nervousness  or  irritation 
Change  in  speech  or  behavior 

RISK  TAKING:  To  what  extent  is  this  individual  willing  to 
risk  negative  consequences  in  order  to  achieve  objectives? 

Additional  points  to  consider: 

Willingness  to  state  a  position  or  take  a  stand 
Willingness  to  make  a  decision  based  on  limited 
information 

Willingness  to  take  an  unpopular  position 
'Willingness  to  assume  responsibility 
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ACQUIRING  INFORMATION:  To  what  extent  is  this  individual 
able  to  obtain  information 

Additional  points  to  consider: 

Number  of  questions  asked 
How  aggressive  is  questioning? 

Is  questioning  adjusted  to  data  received  in  answers 
to  previous  questions? 

How  much  information  is  obtained? 

Follows  through  on  questions 

Ability  to  determine  what  information  is  needed 

ORGANIZING  SKILLS:  To  what  extent  does  this  individual  plan 
and  organize  his/her  activities  and  those  of  others? 

Additional  points  to  consider: 

Are  oral  and  written  reports  logical  and  under¬ 
standable? 

is  use  of  time  efficient? 

Does  the  individual  set  goals  and  work  toward  them? 
Are  activities  structured? 

Are  priorities  assigned  to  different  activities? 

PR0BL5M  SOLVING:  To  what  extent  can  this  individual  identify 
problems  and  find  solutions? 

Additional  points  to  consider: 

Ability  to  sift  out  significant  information  cr  impor¬ 
tant  facets  of  a  problem  and  disregard  nonessentials. 
Ability  to  size  up  a  situation  quickly,  including  the 
people  involved. 

Are  alternative  solutions  identified  and  examined? 
Amount  of  participation  in  group  problem  solving 
situations. 


PHCIDICN  NAKIiTG:  To  ’.vhat  extent  can  this  individual  make 
decisions  based  on  sound  rationale? 

Additional  points  to  consider: 

Ability  to  determine  key  issues 

Is  decision  making  process  logical  and  methodical? 

Are  all  inputs  considered? 

•,7RITTS?r  COMMUNICATIONS  SKILLS :  To  what  extent  can  this  indi¬ 
vidual  effectively  express  ideas  in  a  well  organized,  gram¬ 
matically  correct  written  form? 

Additional  points  to  consider: 

Are  sentences  and  paragraphs  well  structured? 

Are  reports  easy  to  read  and  understand? 

Are  reports  clear  and  concise,  or  are  excess  words 
used? 

Handwriting  should  not  be  a  factor  unless  it  dis¬ 
tracts  from  the  message 

Do  reports  have  an  introduction,  main  points,  and 
conclusions  (when  required)? 
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ft  In  reference  to  the  job  this  individual  currently  occupies, 

i  please  rate  the  importance  of  the  following  managerial  dimen¬ 

sions  with  respect  to  the  effective  completion  of  that  job: 

Use  the  following  rating  scale: 


Critically  Important  5 
Somewhat  Important  4 
Usually  Helpful  3 
Somewhat  Unimportant  2 
Not  Necessary  1 


Oral  Communications  Skills 

Leadership 

Energy 

Forcefulness 

Persuasiveness 

Flexibility 

Empathy 

Stress  Tolerance 

Risk  Taking 

acquiring  Information 

Organizing  Skills 

Problem  Solving 

Decision  Making 

Written  Communications  Skills 


If  required,  refer  back  to  the  questionnaire  for  the  defi¬ 
nitions  of  these  dimensions. 
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DEMOGRAPHIC  DATA 


Some  data  was  collected  and,  subsequently,  not  used 
in  the  achievement  of  the  thesis  objectives.  It  was  felt 
that  this  data  nay  still  be  of  interest  and  is  included  here. 

Subordinates.  The  number  of  professional  subord¬ 
inates  for  whom  the  assessees  were  responsible  for  direct 
supervision  was  obtained  in  the  interview.  The  range  in 
number  of  subordinates  was  0  to  46  with  a  mean  number  of  9.5 
and  a  standard  deviation  of  9.6  subordinates.  Only  nine  of 
those  assessees  had  no  subordinates.  These  were  usually 
technical  specialists  or  program  managers. 

Supervisory  Time.  During  the  interview,  the  super¬ 
visors  were  asked  how  long  they  had  been  supervisors  of  the 
assessees  (as  distinguished  from  being  a  coworker).  This 
time  ranged  from  a  low  of  approximately  2  months  to  a  high 
of  20  years.  The  mean  time  was  2.9  years  with  a  standard 
deviation  of  3.5  years. 

Budget  Responsibility.  The  amount  of  budget  any 
given  assessee  was  responsible  for  was  also  obtained  from 
the  interview.  Overall,  this  amount  ranged  from  $0  to  a  high 
of  $2  billion  with  a  mean  of  350.1  million  and  standard  devi¬ 
ation  of  3271.3  million.  However,  these  figures  are  slightly 
misleading  in  that  42  of  the  assessees  had  no  (30)  budget 
responsibility.  Taking  only  the  25  who  did  have  a  budget 
responsibility,  the  amount  ranged  from  3400  thousand  to 
32  billion  with  a  new  mean  of  3134.2  million. 
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